release v1.4.9

base64 preset support
edited chat message now is marked as Normal
2023-10-27 14:03:28 +08:00 · 2023-10-27 13:35:29 +08:00 · 2023-10-27 13:11:12 +08:00 · 2023-10-27 12:53:20 +08:00 · 2023-10-27 12:13:05 +08:00 · 2023-10-27 11:36:29 +08:00
45 changed files with 683 additions and 719 deletions
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -63,10 +63,10 @@ jobs:
          Expand-Archive ./python-3.10.11-embed-amd64.zip -DestinationPath ./py310
          $content=Get-Content "./py310/python310._pth"; $content | ForEach-Object {if ($_.ReadCount -eq 3) {"Lib\\site-packages"} else {$_}} | Set-Content ./py310/python310._pth
          ./py310/python ./backend-python/get-pip.py
-          ./py310/python -m pip install Cython==0.29.36
+          ./py310/python -m pip install Cython==3.0.4
          Copy-Item -Path "${{ steps.cp310.outputs.python-path }}/../include" -Destination "py310/include" -Recurse
          Copy-Item -Path "${{ steps.cp310.outputs.python-path }}/../libs" -Destination "py310/libs" -Recurse
-          ./py310/python -m pip install cyac==1.7
+          ./py310/python -m pip install cyac==1.9
          git clone https://github.com/josStorer/ai00_rwkv_server --depth=1
          cd ai00_rwkv_server
          cargo build --release
@@ -111,9 +111,6 @@ jobs:
          rm ./backend-python/rwkv_pip/rwkv5.pyd
          rm ./backend-python/rwkv_pip/beta/wkv_cuda.pyd
          rm ./backend-python/get-pip.py
-          sed -i '1,2d' ./backend-golang/wsl_not_windows.go
-          rm ./backend-golang/wsl.go
-          mv ./backend-golang/wsl_not_windows.go ./backend-golang/wsl.go
          make
          mv build/bin/RWKV-Runner build/bin/RWKV-Runner_linux_x64

@@ -145,9 +142,6 @@ jobs:
          rm ./backend-python/rwkv_pip/rwkv5.pyd
          rm ./backend-python/rwkv_pip/beta/wkv_cuda.pyd
          rm ./backend-python/get-pip.py
-          sed -i '' '1,2d' ./backend-golang/wsl_not_windows.go
-          rm ./backend-golang/wsl.go
-          mv ./backend-golang/wsl_not_windows.go ./backend-golang/wsl.go
          make
          cp build/darwin/Readme_Install.txt build/bin/Readme_Install.txt
          cp build/bin/RWKV-Runner.app/Contents/MacOS/RWKV-Runner build/bin/RWKV-Runner_darwin_universal
--- a/.gitignore
+++ b/.gitignore
@@ -18,6 +18,7 @@ __pycache__
 /cmd-helper.bat
 /install-py-dep.bat
 /backend-python/wkv_cuda
+/backend-python/rwkv5
 *.exe
 *.old
 .DS_Store
--- a/CURRENT_CHANGE.md
+++ b/CURRENT_CHANGE.md
@@ -1,9 +1,42 @@
 ## Changes

- latest rwkv-5.2 is now supported (with pre-compiled kernel for windows)
- completion page: add format content button
+### Features
+
+- allow conversation with some document (.pdf, .txt) (Experimental)
+- add `/file-to-text` api
+- allow avatarImg to be local absolute path
+- base64 preset support
+
+### Upgrades
+
+- upgrade to rwkv 0.8.16 (DirectML support; rwkv 5.2 no longer needs to ensure custom cuda kernel enabled)
+- upgrade to webgpu 0.2.2 (WebGPU Mode is now recommended for AMD and Intel
+  Users) (https://github.com/josStorer/ai00_rwkv_server)
+- upgrade python packages
+
+### Improvements
+
+- improve cuda kernel compatibility (compute compatibility 5.3, Jetson Nano, Nvidia 10 Series+)
+- RWKVType now no longer relies on the file name (use emb)
+- improve message interruption and retry for Chat page
+- update sample.jsonl of LoRA finetune
+- update api stop strategy for better custom user_name and assistant_name support
+- edited chat message now is marked as Normal
+- change default World series prefix to User/Assistant
+
+### Chores
+
+- update manifest.json (RWKV-5)
+- update readme and client text description
+- add pip --no-warn-script-location
+- mark rwkv raven series as old model
 - chore

+### Fixes
+
+- fix linux kernel (partial revert 68228a45)
+- fix the `make` command on Linux and macOS, no longer need manual operations on the wsl.go file. (#158, #173, #207)
+
 ## Install

 - Windows: https://github.com/josStorer/RWKV-Runner/blob/master/build/windows/Readme_Install.txt
--- a/README.md
+++ b/README.md
@@ -47,7 +47,9 @@ English | [简体中文](README_ZH.md) | [日本語](README_JA.md)

 </div>

-#### Default configs has enabled custom CUDA kernel acceleration, which is much faster and consumes much less VRAM. If you encounter possible compatibility issues, go to the Configs page and turn off `Use Custom CUDA kernel to Accelerate`.
+#### Tip: You can deploy [backend-python](./backend-python/) on a server and use this program as a client only. Fill in your server address in the Settings `API URL`.
+
+#### Default configs has enabled custom CUDA kernel acceleration, which is much faster and consumes much less VRAM. If you encounter possible compatibility issues (output garbled), go to the Configs page and turn off `Use Custom CUDA kernel to Accelerate`, or try to upgrade your gpu driver.

 #### If Windows Defender claims this is a virus, you can try downloading [v1.3.7_win.zip](https://github.com/josStorer/RWKV-Runner/releases/download/v1.3.7/RWKV-Runner_win.zip) and letting it update automatically to the latest version, or add it to the trusted list (`Windows Security` -> `Virus & threat protection` -> `Manage settings` -> `Exclusions` -> `Add or remove exclusions` -> `Add an exclusion` -> `Folder` -> `RWKV-Runner`).

--- a/README_JA.md
+++ b/README_JA.md
@@ -47,7 +47,9 @@

 </div>

-#### デフォルトの設定はカスタム CUDA カーネルアクセラレーションを有効にしています。互換性の問題が発生する可能性がある場合は、コンフィグページに移動し、`Use Custom CUDA kernel to Accelerate` をオフにしてください。
+#### ヒント：サーバーに[backend-python](./backend-python/)をデプロイし、このプログラムをクライアントとして使用することができます。設定された`API URL`にサーバーアドレスを入力してください。
+
+#### デフォルトの設定はカスタム CUDA カーネルアクセラレーションを有効にしています。互換性の問題 (文字化けを出力する) が発生する可能性がある場合は、コンフィグページに移動し、`Use Custom CUDA kernel to Accelerate` をオフにしてください、あるいは、GPUドライバーをアップグレードしてみてください。

 #### Windows Defender がこれをウイルスだと主張する場合は、[v1.3.7_win.zip](https://github.com/josStorer/RWKV-Runner/releases/download/v1.3.7/RWKV-Runner_win.zip) をダウンロードして最新版に自動更新させるか、信頼済みリストに追加してみてください (`Windows Security` -> `Virus & threat protection` -> `Manage settings` -> `Exclusions` -> `Add or remove exclusions` -> `Add an exclusion` -> `Folder` -> `RWKV-Runner`)。

--- a/README_ZH.md
+++ b/README_ZH.md
@@ -46,7 +46,9 @@ API兼容的接口，这意味着一切ChatGPT客户端都是RWKV客户端。

 </div>

-#### 预设配置已经开启自定义CUDA算子加速，速度更快，且显存消耗更少。如果你遇到可能的兼容性问题，前往配置页面，关闭`使用自定义CUDA算子加速`
+#### 小贴士：你可以在服务器部署[backend-python](./backend-python/)，然后将此程序仅用作客户端，在设置的`API URL`中填入你的服务器地址
+
+#### 预设配置已经开启自定义CUDA算子加速，速度更快，且显存消耗更少。如果你遇到可能的兼容性(输出乱码)问题，前往配置页面，关闭`使用自定义CUDA算子加速`，或更新你的显卡驱动

 #### 如果Windows Defender说这是一个病毒，你可以尝试下载[v1.3.7_win.zip](https://github.com/josStorer/RWKV-Runner/releases/download/v1.3.7/RWKV-Runner_win.zip)，然后让其自动更新到最新版，或添加信任 (`Windows Security` -> `Virus & threat protection` -> `Manage settings` -> `Exclusions` -> `Add or remove exclusions` -> `Add an exclusion` -> `Folder` -> `RWKV-Runner`)

--- a/backend-golang/file.go
+++ b/backend-golang/file.go
@@ -53,12 +53,12 @@ type FileInfo struct {
 	ModTime string `json:"modTime"`
 }

-func (a *App) ReadFileInfo(fileName string) (FileInfo, error) {
+func (a *App) ReadFileInfo(fileName string) (*FileInfo, error) {
 	info, err := os.Stat(a.exDir + fileName)
 	if err != nil {
-		return FileInfo{}, err
+		return nil, err
 	}
-	return FileInfo{
+	return &FileInfo{
 		Name:    info.Name(),
 		Size:    info.Size(),
 		IsDir:   info.IsDir(),
@@ -145,6 +145,20 @@ func (a *App) OpenSaveFileDialogBytes(filterPattern string, defaultFileName stri
 	return path, nil
 }

+// Only return the path of the selected file, because communication between frontend and backend is slow. Use AssetServer Handler to read the file.
+func (a *App) OpenOpenFileDialog(filterPattern string) (string, error) {
+	path, err := wruntime.OpenFileDialog(a.ctx, wruntime.OpenDialogOptions{
+		Filters: []wruntime.FileFilter{{Pattern: filterPattern}},
+	})
+	if err != nil {
+		return "", err
+	}
+	if path == "" {
+		return "", nil
+	}
+	return path, nil
+}
+
 func (a *App) OpenFileFolder(path string, relative bool) error {
 	var absPath string
 	var err error
--- a/backend-golang/rwkv.go
+++ b/backend-golang/rwkv.go
@@ -28,8 +28,7 @@ func (a *App) StartServer(python string, port int, host string, rwkvBeta bool) (

 func (a *App) StartWebGPUServer(port int, host string) (string, error) {
 	args := []string{"./backend-rust/webgpu_server"}
-	args = append(args, "-a", "0", "-t", "backend-rust/assets/rwkv_vocab_v20230424.json",
-		"--port", strconv.Itoa(port), "--ip", host)
+	args = append(args, "--port", strconv.Itoa(port), "--ip", host)
 	return Cmd(args...)
 }

@@ -149,9 +148,9 @@ func (a *App) InstallPyDep(python string, cnMirror bool) (string, error) {

 	if runtime.GOOS == "windows" {
 		ChangeFileLine("./py310/python310._pth", 3, "Lib\\site-packages")
-		installScript := python + " ./backend-python/get-pip.py -i https://pypi.tuna.tsinghua.edu.cn/simple\n" +
-			python + " -m pip install torch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 --index-url https://download.pytorch.org/whl/cu117\n" +
-			python + " -m pip install -r ./backend-python/requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple\n" +
+		installScript := python + " ./backend-python/get-pip.py -i https://pypi.tuna.tsinghua.edu.cn/simple --no-warn-script-location\n" +
+			python + " -m pip install torch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 --index-url https://download.pytorch.org/whl/cu117 --no-warn-script-location\n" +
+			python + " -m pip install -r ./backend-python/requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple --no-warn-script-location\n" +
 			"exit"
 		if !cnMirror {
 			installScript = strings.Replace(installScript, " -i https://pypi.tuna.tsinghua.edu.cn/simple", "", -1)
--- a/backend-golang/wsl_not_windows.go
+++ b/backend-golang/wsl_not_windows.go
--- a/backend-golang/wsl_windows.go
+++ b/backend-golang/wsl_windows.go
--- a/backend-python/convert_safetensors.py
+++ b/backend-python/convert_safetensors.py
@@ -18,20 +18,31 @@ parser.add_argument(
 args = parser.parse_args()


-def convert_file(
-    pt_filename: str,
-    sf_filename: str,
-):
+def rename_key(rename, name):
+    for k, v in rename.items():
+        if k in name:
+            name = name.replace(k, v)
+    return name
+
+
+def convert_file(pt_filename: str, sf_filename: str, transpose_names=[], rename={}):
    loaded = torch.load(pt_filename, map_location="cpu")
    if "state_dict" in loaded:
        loaded = loaded["state_dict"]

    loaded = {k: v.clone().half() for k, v in loaded.items()}
-    for k, v in loaded.items():
-        print(f"{k}\t{v.shape}\t{v.dtype}")
+    # for k, v in loaded.items():
+    #     print(f'{k}\t{v.shape}\t{v.dtype}')

    # For tensors to be contiguous
-    loaded = {k: v.contiguous() for k, v in loaded.items()}
+    for k, v in loaded.items():
+        for transpose_name in transpose_names:
+            if transpose_name in k:
+                loaded[k] = v.transpose(0, 1)
+    loaded = {rename_key(rename, k).lower(): v.contiguous() for k, v in loaded.items()}
+
+    for k, v in loaded.items():
+        print(f"{k}\t{v.shape}\t{v.dtype}")

    dirname = os.path.dirname(sf_filename)
    os.makedirs(dirname, exist_ok=True)
@@ -46,7 +57,12 @@ def convert_file(

 if __name__ == "__main__":
    try:
-        convert_file(args.input, args.output)
+        convert_file(
+            args.input,
+            args.output,
+            ["lora_A"],
+            {"time_faaaa": "time_first", "lora_A": "lora.0", "lora_B": "lora.1"},
+        )
        print(f"Saved to {args.output}")
    except Exception as e:
        with open("error.txt", "w") as f:
--- a/backend-python/dep_check.py
+++ b/backend-python/dep_check.py
@@ -1,3 +1,5 @@
+import multipart
+import fitz
 import safetensors
 import midi2audio
 import mido
@@ -9,6 +11,7 @@ import GPUtil

 import torch
 import rwkv
+import langchain
 import numpy
 import tokenizers
 import fastapi
--- a/backend-python/main.py
+++ b/backend-python/main.py
@@ -2,10 +2,12 @@ import time

 start_time = time.time()

+import setuptools  # avoid warnings
 import os
 import sys
 import argparse
 from typing import Sequence
+from contextlib import asynccontextmanager

 sys.path.append(os.path.dirname(os.path.realpath(__file__)))

@@ -18,10 +20,17 @@ from utils.rwkv import *
 from utils.torch import *
 from utils.ngrok import *
 from utils.log import log_middleware
-from routes import completion, config, state_cache, midi, misc
+from routes import completion, config, state_cache, midi, misc, file_process
 import global_var

-app = FastAPI(dependencies=[Depends(log_middleware)])
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    init()
+    yield
+
+
+app = FastAPI(lifespan=lifespan, dependencies=[Depends(log_middleware)])

 app.add_middleware(
    CORSMiddleware,
@@ -34,11 +43,11 @@ app.add_middleware(
 app.include_router(completion.router)
 app.include_router(config.router)
 app.include_router(midi.router)
+app.include_router(file_process.router)
 app.include_router(misc.router)
 app.include_router(state_cache.router)


-@app.on_event("startup")
 def init():
    global_var.init()
    cmd_params = os.environ["RWKV_RUNNER_PARAMS"]
--- a/backend-python/requirements.txt
+++ b/backend-python/requirements.txt
--- a/backend-python/requirements_without_cyac.txt
+++ b/backend-python/requirements_without_cyac.txt
--- a/backend-python/routes/completion.py
+++ b/backend-python/routes/completion.py
@@ -43,16 +43,18 @@ class ChatCompletionBody(ModelConfigBody):
    model: Union[str, None] = "rwkv"
    stream: bool = False
    stop: Union[str, List[str], None] = default_stop
-    user_name: Union[str, None] = Field(None, description="Internal user name")
+    user_name: Union[str, None] = Field(
+        None, description="Internal user name", min_length=1
+    )
    assistant_name: Union[str, None] = Field(
-        None, description="Internal assistant name"
+        None, description="Internal assistant name", min_length=1
    )
    presystem: bool = Field(
        True, description="Whether to insert default system prompt at the beginning"
    )

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "messages": [
                    {"role": Role.User.value, "content": "hello", "raw": False}
@@ -79,7 +81,7 @@ class CompletionBody(ModelConfigBody):
    stop: Union[str, List[str], None] = None

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "prompt": "The following is an epic science fiction masterpiece that is immortalized, "
                + "with delicate descriptions and grand depictions of interstellar civilization wars.\nChapter 1.\n",
@@ -317,11 +319,13 @@ The following is a coherent verbose detailed conversation between a girl named {
        completion_text += append_message + "\n\n"
    completion_text += f"{bot}{interface}"

+    user_code = model.pipeline.decode([model.pipeline.encode(user)[0]])
+    bot_code = model.pipeline.decode([model.pipeline.encode(bot)[0]])
    if type(body.stop) == str:
-        body.stop = [body.stop, f"\n\n{user}", f"\n\n{bot}"]
+        body.stop = [body.stop, f"\n\n{user_code}", f"\n\n{bot_code}"]
    elif type(body.stop) == list:
-        body.stop.append(f"\n\n{user}")
-        body.stop.append(f"\n\n{bot}")
+        body.stop.append(f"\n\n{user_code}")
+        body.stop.append(f"\n\n{bot_code}")
    elif body.stop is None:
        body.stop = default_stop

@@ -373,7 +377,7 @@ class EmbeddingsBody(BaseModel):
    fast_mode: bool = False

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "input": "a big apple",
                "model": "rwkv",
--- a/backend-python/routes/config.py
+++ b/backend-python/routes/config.py
@@ -10,22 +10,6 @@ import global_var
 router = APIRouter()


-def get_tokens_path(model_path: str):
-    model_path = model_path.lower()
-    tokenizer_dir = f"{pathlib.Path(__file__).parent.parent.resolve()}/rwkv_pip/"
-
-    default_tokens_path = tokenizer_dir + "20B_tokenizer.json"
-
-    if "raven" in model_path:
-        return default_tokens_path
-    elif "world" in model_path:
-        return "rwkv_vocab_v20230424"
-    elif "midi" in model_path:
-        return tokenizer_dir + "tokenizer-midi.json"
-    else:
-        return default_tokens_path
-
-
 class SwitchModelBody(BaseModel):
    model: str
    strategy: str
@@ -33,7 +17,7 @@ class SwitchModelBody(BaseModel):
    customCuda: bool = False

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "model": "models/RWKV-4-World-3B-v1-20230619-ctx4096.pth",
                "strategy": "cuda fp16",
@@ -67,25 +51,10 @@ def switch_model(body: SwitchModelBody, response: Response, request: Request):
    os.environ["RWKV_CUDA_ON"] = "1" if body.customCuda else "0"

    global_var.set(global_var.Model_Status, global_var.ModelStatus.Loading)
-    tokenizer = (
-        get_tokens_path(body.model)
-        if body.tokenizer is None or body.tokenizer == ""
-        else body.tokenizer
-    )
    try:
        global_var.set(
            global_var.Model,
-            TextRWKV(
-                model=body.model,
-                strategy=body.strategy,
-                tokens_path=tokenizer,
-            )
-            if "midi" not in body.model.lower()
-            else MusicRWKV(
-                model=body.model,
-                strategy=body.strategy,
-                tokens_path=tokenizer,
-            ),
+            RWKV(model=body.model, strategy=body.strategy, tokenizer=body.tokenizer),
        )
    except Exception as e:
        print(e)
--- a/backend-python/routes/file_process.py
+++ b/backend-python/routes/file_process.py
@@ -0,0 +1,79 @@
+import os
+from fastapi import (
+    APIRouter,
+    HTTPException,
+    status,
+    Depends,
+    File,
+    UploadFile,
+)
+from pydantic import BaseModel
+from typing import Iterator
+
+router = APIRouter()
+
+
+class FileToTextParams(BaseModel):
+    file_name: str
+    file_encoding: str = "utf-8"
+
+
+@router.post("/file-to-text", tags=["File Process"])
+async def file_to_text(
+    params: FileToTextParams = Depends(), file_data: UploadFile = File(...)
+):
+    from langchain.schema import Document
+    from langchain.document_loaders.blob_loaders import Blob
+
+    # from langchain
+    def parse_text(blob: Blob) -> Iterator[Document]:
+        yield Document(page_content=blob.as_string(), metadata={"source": blob.source})
+
+    # from langchain
+    def parse_pdf(blob: Blob) -> Iterator[Document]:
+        import fitz
+
+        with blob.as_bytes_io() as stream:
+            doc = fitz.Document(stream=stream)
+
+            yield from [
+                Document(
+                    page_content=page.get_text(),
+                    metadata=dict(
+                        {
+                            "source": blob.source,
+                            "file_path": blob.source,
+                            "page": page.number,
+                            "total_pages": len(doc),
+                        },
+                        **{
+                            k: doc.metadata[k]
+                            for k in doc.metadata
+                            if type(doc.metadata[k]) in [str, int]
+                        },
+                    ),
+                )
+                for page in doc
+            ]
+
+    file_parsers = {".txt": parse_text, ".pdf": parse_pdf}
+
+    file_name = file_data.filename or params.file_name
+    file_ext = os.path.splitext(file_name)[-1]
+
+    if file_ext not in file_parsers:
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, "file type not supported")
+
+    try:
+        pages: Iterator[Document] = file_parsers[file_ext](
+            Blob.from_data(
+                await file_data.read(),
+                encoding=params.file_encoding,
+                path=file_name,
+            )
+        )
+        pages = list(pages)
+    except Exception as e:
+        raise HTTPException(status.HTTP_400_BAD_REQUEST, f"{e}")
+
+    return {"pages": pages}
--- a/backend-python/routes/midi.py
+++ b/backend-python/routes/midi.py
@@ -12,7 +12,7 @@ class TextToMidiBody(BaseModel):
    text: str

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "text": "p:24:a p:2a:a p:31:a p:39:a p:3b:a p:45:a b:26:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:24:0 p:2a:0 p:31:0 p:39:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:26:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:2e:a p:3b:a p:45:a b:26:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:2e:0 p:3b:0 p:45:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:2e:a p:3b:a p:45:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:2e:0 p:3b:0 p:45:0 b:26:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:26:a p:2a:a p:3b:a p:45:a t14 p:26:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a b:26:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:2a:0 p:3b:0 p:45:0 b:26:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:2d:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 b:2d:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:24:a p:2e:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:24:0 p:2e:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:26:a p:2a:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:26:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:26:a p:2e:a p:31:a p:39:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:26:0 p:2e:0 p:31:0 p:39:0 p:3b:0 p:45:0 b:21:0 t2 p:26:a p:2e:a p:31:a p:39:a p:3b:a p:45:a b:21:a t14 p:26:0 p:2e:0 p:31:0 p:39:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:24:a p:2a:a p:31:a p:39:a p:3b:a p:45:a b:1f:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:24:0 p:2a:0 p:31:0 p:39:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:1f:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:2e:a p:3b:a p:45:a b:1f:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:2e:0 p:3b:0 p:45:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:2e:a p:3b:a p:45:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:2e:0 p:3b:0 p:45:0 b:1f:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:26:a p:2a:a p:3b:a p:45:a t14 p:26:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a b:1f:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:2a:0 p:3b:0 p:45:0 b:1f:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:1f:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 b:1f:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:24:a p:2e:a p:3b:a p:45:a b:26:a g:39:a g:39:a g:3e:a g:3e:a g:42:a g:42:a pi:39:a pi:3e:a pi:42:a t14 p:24:0 p:2e:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0",
            }
@@ -36,7 +36,7 @@ class TxtToMidiBody(BaseModel):
    midi_path: str

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "txt_path": "midi/sample.txt",
                "midi_path": "midi/sample.mid",
@@ -66,7 +66,7 @@ class MidiToWavBody(BaseModel):
    sound_font_path: str = "assets/default_sound_font.sf2"

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "midi_path": "midi/sample.mid",
                "wav_path": "midi/sample.wav",
@@ -96,7 +96,7 @@ class TextToWavBody(BaseModel):
    sound_font_path: str = "assets/default_sound_font.sf2"

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "text": "p:24:a p:2a:a p:31:a p:39:a p:3b:a p:45:a b:26:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:24:0 p:2a:0 p:31:0 p:39:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:26:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:2e:a p:3b:a p:45:a b:26:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:2e:0 p:3b:0 p:45:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:2e:a p:3b:a p:45:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:2e:0 p:3b:0 p:45:0 b:26:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:26:a p:2a:a p:3b:a p:45:a t14 p:26:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a b:26:a g:3e:a g:3e:a g:42:a g:42:a g:45:a g:45:a pi:3e:a pi:42:a pi:45:a t14 p:2a:0 p:3b:0 p:45:0 b:26:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:2d:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 b:2d:0 g:3e:0 g:3e:0 g:42:0 g:42:0 g:45:0 g:45:0 pi:3e:0 pi:42:0 pi:45:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:24:a p:2e:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:24:0 p:2e:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:26:a p:2a:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:26:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:26:a p:2e:a p:31:a p:39:a p:3b:a p:45:a b:21:a g:39:a g:39:a g:3d:a g:3d:a g:40:a g:40:a pi:39:a pi:3d:a pi:40:a t14 p:26:0 p:2e:0 p:31:0 p:39:0 p:3b:0 p:45:0 b:21:0 t2 p:26:a p:2e:a p:31:a p:39:a p:3b:a p:45:a b:21:a t14 p:26:0 p:2e:0 p:31:0 p:39:0 p:3b:0 p:45:0 b:21:0 g:39:0 g:39:0 g:3d:0 g:3d:0 g:40:0 g:40:0 pi:39:0 pi:3d:0 pi:40:0 t2 p:24:a p:2a:a p:31:a p:39:a p:3b:a p:45:a b:1f:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:24:0 p:2a:0 p:31:0 p:39:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0 p:45:0 b:1f:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:2e:a p:3b:a p:45:a b:1f:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:2e:0 p:3b:0 p:45:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:2e:a p:3b:a p:45:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:2e:0 p:3b:0 p:45:0 b:1f:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:26:a p:2a:a p:3b:a p:45:a t14 p:26:0 p:2a:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a b:1f:a g:3b:a g:3b:a g:3e:a g:3e:a g:43:a g:43:a pi:3b:a pi:3e:a pi:43:a t14 p:2a:0 p:3b:0 p:45:0 b:1f:0 t2 p:24:a p:2a:a p:3b:a p:45:a b:1f:a t14 p:24:0 p:2a:0 p:3b:0 p:45:0 b:1f:0 g:3b:0 g:3b:0 g:3e:0 g:3e:0 g:43:0 g:43:0 pi:3b:0 pi:3e:0 pi:43:0 t2 p:24:a p:2e:a p:3b:a p:45:a b:26:a g:39:a g:39:a g:3e:a g:3e:a g:42:a g:42:a pi:39:a pi:3e:a pi:42:a t14 p:24:0 p:2e:0 p:3b:0 p:45:0 t2 p:2a:a p:3b:a p:45:a t14 p:2a:0 p:3b:0",
                "wav_name": "sample",
--- a/backend-python/rwkv_pip/beta/model.py
+++ b/backend-python/rwkv_pip/beta/model.py
@@ -94,7 +94,7 @@ if os.environ.get("RWKV_CUDA_ON") == "1":
                f"{current_path}/cuda/att_one_v5.cu",
            ],
            verbose=True,
-            extra_ldflags=["cublas.lib"],
+            extra_ldflags=["cublas.lib" if os.name == "nt" else ""],
            extra_cuda_cflags=[
                "-t 4",
                "-std=c++17",
--- a/backend-python/rwkv_pip/beta/wkv_cuda.pyd
+++ b/backend-python/rwkv_pip/beta/wkv_cuda.pyd
--- a/backend-python/rwkv_pip/cuda/att_one.cu
+++ b/backend-python/rwkv_pip/cuda/att_one.cu
@@ -1,124 +0,0 @@
-#include "ATen/ATen.h"
-#include <cuda_fp16.h>
-#include <cuda_runtime.h>
-#include <torch/extension.h>
-
-#include "element_wise.h"
-#include "util.h"
-
-// Equivalent Python code:
-// ww = t_first + k
-// p = torch.maximum(pp, ww)
-// e1 = torch.exp(pp - p)
-// e2 = torch.exp(ww - p)
-// wkv = ((e1 * aa + e2 * v) / (e1 * bb + e2)).to(dtype=x.dtype)
-// ww = t_decay + pp
-// p = torch.maximum(ww, k)
-// e1 = torch.exp(ww - p)
-// e2 = torch.exp(k - p)
-// t1 = e1 * aa + e2 * v
-// t2 = e1 * bb + e2
-// r = r * wkv
-// return t1, t2, p, r
-struct WkvForwardOne {
-  const float *t_first;
-  const float *k;
-  const float *pp;
-  const float *aa;
-  const float *bb;
-  const float *t_decay;
-  const float *v;
-  /* out */ float *t1;
-  /* out */ float *t2;
-  /* out */ float *p;
-  /* in & out */ half *r;
-
-  __device__ void operator()(int i) const {
-    float ww = t_first[i] + k[i];
-    float pp_ = pp[i];
-    float p_ = (pp_ > ww) ? pp_ : ww;
-    float e1 = expf(pp_ - p_);
-    float e2 = expf(ww - p_);
-    float aa_ = aa[i];
-    float bb_ = bb[i];
-    float v_ = v[i];
-    r[i] = __hmul(r[i], __float2half(((e1 * aa_ + e2 * v_) / (e1 * bb_ + e2))));
-    ww = t_decay[i] + pp_;
-    float k_ = k[i];
-    p_ = (ww > k_) ? ww : k_;
-    e1 = expf(ww - p_);
-    e2 = expf(k_ - p_);
-    t1[i] = e1 * aa_ + e2 * v_;
-    t2[i] = e1 * bb_ + e2;
-    p[i] = p_;
-  }
-};
-
-/*
-   Equivalent Python code:
-   kx = xx * k_mix + sx * (1 - k_mix)
-   vx = xx * v_mix + sx * (1 - v_mix)
-   rx = xx * r_mix + sx * (1 - r_mix)
-*/
-
-struct Mix {
-  const half *xx;
-  const half *sx;
-  const half *k_mix;
-  const half *v_mix;
-  const half *r_mix;
-  /* out */ half *kx;
-  /* out */ half *vx;
-  /* out */ half *rx;
-
-  __device__ void operator()(int i) const {
-    half xx_ = xx[i];
-    half sx_ = sx[i];
-    half k_mix_ = k_mix[i];
-    half v_mix_ = v_mix[i];
-    half r_mix_ = r_mix[i];
-    kx[i] = __hadd(__hmul(xx_, k_mix_),
-                   __hmul(sx_, __hsub(__float2half(1), k_mix_)));
-    vx[i] = __hadd(__hmul(xx_, v_mix_),
-                   __hmul(sx_, __hsub(__float2half(1), v_mix_)));
-    rx[i] = __hadd(__hmul(xx_, r_mix_),
-                   __hmul(sx_, __hsub(__float2half(1), r_mix_)));
-  }
-};
-
-using torch::Tensor;
-
-void gemm_fp16_cublas(Tensor a, Tensor b, Tensor c);
-
-Tensor att_one(Tensor x, Tensor ln_w, Tensor ln_b, Tensor sx, Tensor k_mix,
-             Tensor v_mix, Tensor r_mix, Tensor kw,
-             /* imm */ Tensor kx, Tensor vw, /* imm */ Tensor vx, Tensor rw,
-             /* imm */ Tensor rx, Tensor ow, Tensor t_first,
-             /* imm */ Tensor k, Tensor pp, Tensor ww, Tensor aa, Tensor bb,
-             Tensor t_decay, /* imm */ Tensor v, /* in & out */ Tensor r,
-             /* out */ Tensor x_plus_out, /* out */ Tensor t1,
-             /* out */ Tensor t2, /* out */ Tensor p) {
-  Tensor xx = at::layer_norm(x, {x.size(-1)}, ln_w, ln_b);
-  element_wise(Mix{data_ptr<half>(xx), data_ptr<half>(sx),
-                   data_ptr<half>(k_mix), data_ptr<half>(v_mix),
-                   data_ptr<half>(r_mix), data_ptr<half>(kx),
-                   data_ptr<half>(vx), data_ptr<half>(rx)},
-               x.numel());
-
-  gemm_fp16_cublas(kx, kw, k);
-  gemm_fp16_cublas(vx, vw, v);
-  gemm_fp16_cublas(rx, rw, r);
-  at::sigmoid_(r);
-
-  element_wise(WkvForwardOne{data_ptr<float>(t_first), data_ptr<float>(k),
-                             data_ptr<float>(pp), data_ptr<float>(aa),
-                             data_ptr<float>(bb), data_ptr<float>(t_decay),
-                             data_ptr<float>(v), data_ptr<float>(t1),
-                             data_ptr<float>(t2), data_ptr<float>(p),
-                             data_ptr<half>(r)},
-               x.numel());
-
-  gemm_fp16_cublas(r, ow, x_plus_out);
-  x_plus_out += x;
-  return xx;
-}
--- a/backend-python/rwkv_pip/cuda/att_seq.cu
+++ b/backend-python/rwkv_pip/cuda/att_seq.cu
@@ -1,179 +0,0 @@
-#include "ATen/ATen.h"
-#include <cuda_fp16.h>
-#include <cuda_runtime.h>
-#include <torch/extension.h>
-
-#include "util.h"
-#include "element_wise.h"
-
-using torch::Tensor;
-
-void gemm_fp16_cublas(Tensor a, Tensor b, Tensor c);
-void gemm_fp16_cublas(const void *a, const void *b, void *c, int m,
-                      int n, int k, bool output_fp32);
-
-// based on `kernel_wkv_forward`, fusing more operations
-__global__ void kernel_wkv_forward_new(
-    const int B, const int T, const int C, const float *__restrict__ const _w,
-    const float *__restrict__ const _u, const float *__restrict__ const _k,
-    const float *__restrict__ const _v, const half *__restrict__ const r,
-    half *__restrict__ const _y, float *__restrict__ const _aa,
-    float *__restrict__ const _bb, float *__restrict__ const _pp) {
-  const int idx = blockIdx.x * blockDim.x + threadIdx.x;
-  const int _b = idx / C;
-  const int _c = idx % C;
-  const int _offset = _b * T * C + _c;
-  const int _state_offset = _b * C + _c;
-
-  float u = _u[_c];
-  float w = _w[_c];
-  const float *__restrict__ const k = _k + _offset;
-  const float *__restrict__ const v = _v + _offset;
-  half *__restrict__ const y = _y + _offset;
-
-  float aa = _aa[_state_offset];
-  float bb = _bb[_state_offset];
-  float pp = _pp[_state_offset];
-  for (int i = 0; i < T; i++) {
-    const int ii = i * C;
-    const float kk = k[ii];
-    const float vv = v[ii];
-    float ww = u + kk;
-    float p = max(pp, ww);
-    float e1 = exp(pp - p);
-    float e2 = exp(ww - p);
-    y[ii] = __float2half((e1 * aa + e2 * vv) / (e1 * bb + e2));
-    ww = w + pp;
-    p = max(ww, kk);
-    e1 = exp(ww - p);
-    e2 = exp(kk - p);
-    aa = e1 * aa + e2 * vv;
-    bb = e1 * bb + e2;
-    pp = p;
-  }
-  _aa[_state_offset] = aa;
-  _bb[_state_offset] = bb;
-  _pp[_state_offset] = pp;
-}
-
-void cuda_wkv_forward_new(int B, int T, int C, float *w, float *u, float *k,
-                          float *v, half *r, half *y, float *aa, float *bb,
-                          float *pp) {
-  dim3 threadsPerBlock(min(C, 32));
-  assert(B * C % threadsPerBlock.x == 0);
-  dim3 numBlocks(B * C / threadsPerBlock.x);
-  kernel_wkv_forward_new<<<numBlocks, threadsPerBlock>>>(B, T, C, w, u, k, v, r,
-                                                         y, aa, bb, pp);
-}
-
-__global__ void _att_mix(const half *xx, const half *sx, const half *k_mix,
-                        const half *v_mix, const half *r_mix,
-                        const int outer_size, const int inner_size, half *kx,
-                        half *vx, half *rx) {
-  for (int idx2 = blockIdx.x * blockDim.x + threadIdx.x; idx2 < inner_size;
-       idx2 += blockDim.x * gridDim.x) {
-    half k_mix_ = k_mix[idx2];
-    half v_mix_ = v_mix[idx2];
-    half r_mix_ = r_mix[idx2];
-    for (int row = 0; row < outer_size; ++row) {
-      int idx1 = row * inner_size + idx2;
-      half xx_ = xx[idx1];
-      half sx_ = sx[idx1];
-      kx[idx1] = __hadd(__hmul(xx_, k_mix_),
-                        __hmul(sx_, __hsub(__float2half(1), k_mix_)));
-      vx[idx1] = __hadd(__hmul(xx_, v_mix_),
-                        __hmul(sx_, __hsub(__float2half(1), v_mix_)));
-      rx[idx1] = __hadd(__hmul(xx_, r_mix_),
-                        __hmul(sx_, __hsub(__float2half(1), r_mix_)));
-    }
-  }
-}
-
-void att_mix(const half *xx, const half *sx, const half *k_mix,
-            const half *v_mix, const half *r_mix, const int outer_size,
-            const int inner_size, half *kx, half *vx, half *rx) {
-  // 256 is good enough on most GPUs
-  const int32_t BLOCK_SIZE = 256;
-  assert(inner_size % BLOCK_SIZE == 0);
-  _att_mix<<<inner_size / BLOCK_SIZE, BLOCK_SIZE>>>(
-      xx, sx, k_mix, v_mix, r_mix, outer_size, inner_size, kx, vx, rx);
-}
-
-struct InplaceSigmoid {
-  __device__ __forceinline__ half operator()(int i) const {
-    ptr[i] = __float2half(1.0 / (1.0 + exp(-__half2float(ptr[i]))));
-  }
-  half *ptr;
-};
-
-struct InplaceMul {
-  __device__ __forceinline__ half operator()(int i) const {
-    y[i] = __hmul(x[i], y[i]);
-  }
-  half *y;
-  half *x;
-};
-
-/*
-   Equivalent Python code:
-
-   xx = F.layer_norm(x, (x.shape[-1],), weight=ln_w, bias=ln_b)
-   sx = torch.cat((sx.unsqueeze(0), xx[:-1,:]))
-   kx = xx * k_mix + sx * (1 - k_mix)
-   vx = xx * v_mix + sx * (1 - v_mix)
-   rx = xx * r_mix + sx * (1 - r_mix)
-
-   r = torch.sigmoid(gemm(rx, rw))
-   k = gemm(kx, kw, output_dtype=torch.float32)
-   v = gemm(vx, vw, output_dtype=torch.float32)
-
-   T = x.shape[0]
-   for t in range(T):
-       kk = k[t]
-       vv = v[t]
-       ww = t_first + kk
-       p = torch.maximum(pp, ww)
-       e1 = torch.exp(pp - p)
-       e2 = torch.exp(ww - p)
-       sx[t] = ((e1 * aa + e2 * vv) / (e1 * bb + e2)).to(dtype=x.dtype)
-       ww = t_decay + pp
-       p = torch.maximum(ww, kk)
-       e1 = torch.exp(ww - p)
-       e2 = torch.exp(kk - p)
-       aa = e1 * aa + e2 * vv
-       bb = e1 * bb + e2
-       pp = p
-   out = gemm(r * sx, ow)
-   return x + out, xx[-1,:], aa, bb, pp
-*/
-Tensor att_seq(Tensor x, Tensor sx, Tensor ln_w, Tensor ln_b, Tensor k_mix,
-               Tensor v_mix, Tensor r_mix, Tensor kw, Tensor vw, Tensor rw,
-               Tensor ow, Tensor t_first, Tensor pp, Tensor aa, Tensor bb,
-               Tensor t_decay, /* imm */ Tensor buf, /* out */ Tensor x_plus_out) {
-  Tensor xx = at::layer_norm(x, {x.size(-1)}, ln_w, ln_b);
-  sx = at::cat({sx.unsqueeze(0), xx.slice(0, 0, -1)}, 0);
-  char* buf_ptr = (char*)buf.data_ptr();
-  half* kx = (half*)buf_ptr;
-  half* vx = kx + x.numel();
-  half* rx = vx + x.numel();
-  half* wkv_y = rx + x.numel();
-  att_mix(data_ptr<half>(xx), data_ptr<half>(sx), data_ptr<half>(k_mix),
-         data_ptr<half>(v_mix), data_ptr<half>(r_mix), xx.size(0), xx.size(1),
-         kx, vx, rx);
-  float* k = reinterpret_cast<float*>(wkv_y + x.numel());
-  float* v = k + x.size(0) * kw.size(1);
-  half* r = reinterpret_cast<half*>(v + x.size(0) * vw.size(1));
-
-  gemm_fp16_cublas(kx, kw.data_ptr(), k, x.size(0), kw.size(1), kw.size(0), true);
-  gemm_fp16_cublas(vx, vw.data_ptr(), v, x.size(0), vw.size(1), vw.size(0), true);
-  gemm_fp16_cublas(rx, rw.data_ptr(), r, x.size(0), rw.size(1), rw.size(0), false);
-  element_wise(InplaceSigmoid{r}, x.size(0) * rw.size(1));
-  cuda_wkv_forward_new(1, x.size(0), x.size(1), data_ptr<float>(t_decay),
-                       data_ptr<float>(t_first), k, v, r,
-                       wkv_y, data_ptr<float>(aa),
-                       data_ptr<float>(bb), data_ptr<float>(pp));
-  element_wise(InplaceMul{wkv_y, r}, x.numel());
-  gemm_fp16_cublas(wkv_y, ow.data_ptr(), x_plus_out.data_ptr(), x.size(0), ow.size(1), ow.size(0), false);
-  x_plus_out += x;
-  return xx;
-}
--- a/backend-python/rwkv_pip/cuda/element_wise.h
+++ b/backend-python/rwkv_pip/cuda/element_wise.h
@@ -1,21 +0,0 @@
-#include <cassert>
-#include <cstddef>
-#include <cstdint>
-
-template <typename Func> __global__ void _element_wise(Func func, int n) {
-  for (int i = blockIdx.x * blockDim.x + threadIdx.x; i < n;
-       i += blockDim.x * gridDim.x) {
-    func(i);
-  }
-}
-
-// NOTE: packed data type (e.g. float4) is a overkill for current sizes
-// (4096 in 7B model and 768 in 0.1B model),
-// and is not faster than the plain float version.
-template <typename Func>
-void element_wise(Func func, int n) {
-  // 256 is good enough on most GPUs
-  const int32_t BLOCK_SIZE = 256;
-  assert(n % BLOCK_SIZE == 0);
-  _element_wise<<<n / BLOCK_SIZE, BLOCK_SIZE>>>(func, n);
-}
--- a/backend-python/rwkv_pip/cuda/ffn.cu
+++ b/backend-python/rwkv_pip/cuda/ffn.cu
@@ -1,165 +0,0 @@
-#include "ATen/ATen.h"
-#include <cuda_fp16.h>
-#include <cuda_runtime.h>
-#include <torch/extension.h>
-
-#include "element_wise.h"
-#include "util.h"
-
-using torch::Tensor;
-
-void gemm_fp16_cublas(const void *a, const void *b, void *c, int ori_m,
-                      int ori_n, int ori_k, bool output_fp32);
-
-__global__ void _ffn_seq_mix(const half *xx, const half *sx, const half *k_mix,
-                             const half *r_mix, const int outer_size,
-                             const int inner_size, half *kx, half *rx) {
-  for (int idx2 = blockIdx.x * blockDim.x + threadIdx.x; idx2 < inner_size;
-       idx2 += blockDim.x * gridDim.x) {
-    half k_mix_ = k_mix[idx2];
-    half r_mix_ = r_mix[idx2];
-    for (int row = 0; row < outer_size; ++row) {
-      int idx1 = row * inner_size + idx2;
-      half xx_ = xx[idx1];
-      half sx_ = sx[idx1];
-      kx[idx1] = __hadd(__hmul(xx_, k_mix_),
-                        __hmul(sx_, __hsub(__float2half(1), k_mix_)));
-      rx[idx1] = __hadd(__hmul(xx_, r_mix_),
-                        __hmul(sx_, __hsub(__float2half(1), r_mix_)));
-    }
-  }
-}
-
-void ffn_seq_mix(const half *xx, const half *sx, const half *k_mix,
-                 const half *r_mix, const int outer_size, const int inner_size,
-                 half *kx, half *rx) {
-  // 256 is good enough on most GPUs
-  const int32_t BLOCK_SIZE = 256;
-  assert(inner_size % BLOCK_SIZE == 0);
-  _ffn_seq_mix<<<inner_size / BLOCK_SIZE, BLOCK_SIZE>>>(
-      xx, sx, k_mix, r_mix, outer_size, inner_size, kx, rx);
-}
-
-struct InplaceSigmoid {
-  __device__ __forceinline__ void operator()(int i) const {
-    ptr[i] = __float2half(1.0 / (1.0 + exp(-__half2float(ptr[i]))));
-  }
-  half *ptr;
-};
-
-struct InplaceReLUAndSquare {
-  __device__ __forceinline__ void operator()(int i) const {
-    // __hmax is not defined in old cuda
-    if (__hgt(ptr[i], __float2half(0))) {
-      ptr[i] = __hmul(ptr[i], ptr[i]);
-    } else {
-      ptr[i] = __float2half(0);
-    }
-  }
-  half *ptr;
-};
-
-struct InplaceFma {
-  __device__ __forceinline__ void operator()(int i) const {
-    a[i] = __hfma(a[i], b[i], c[i]);
-  }
-  half *a;
-  const half *b;
-  const half *c;
-};
-
-/*
-   Equivalent Python code:
-
-   xx = F.layer_norm(x, (x.shape[-1],), weight=ln_w, bias=ln_b)
-   sx = torch.cat((sx.unsqueeze(0), xx[:-1,:]))
-   kx = xx * k_mix + sx * (1 - k_mix)
-   rx = xx * r_mix + sx * (1 - r_mix)
-
-   r = torch.sigmoid(gemm(rx, rw))
-   vx = torch.square(torch.relu(gemm(kx, kw)))
-   out = r * gemm(vx, vw)
-   return x + out, xx[-1,:]
-*/
-Tensor ffn_seq(Tensor x, Tensor sx, Tensor ln_w, Tensor ln_b, Tensor k_mix,
-               Tensor r_mix, Tensor kw, Tensor vw, Tensor rw,
-               /* imm */ Tensor buf,
-               /* out */ Tensor x_plus_out) {
-  Tensor xx = at::layer_norm(x, {x.size(-1)}, ln_w, ln_b);
-  sx = at::cat({sx.unsqueeze(0), xx.slice(0, 0, -1)}, 0);
-  char *buf_ptr = (char *)buf.data_ptr();
-  half *kx = (half *)buf_ptr;
-  half *rx = kx + x.numel();
-  half *vx = rx + x.numel();
-  half *r = vx + x.size(0) * kw.size(1);
-  ffn_seq_mix(data_ptr<half>(xx), data_ptr<half>(sx), data_ptr<half>(k_mix),
-              data_ptr<half>(r_mix), xx.size(0), xx.size(1), kx, rx);
-
-  gemm_fp16_cublas(rx, rw.data_ptr(), r, x.size(0), rw.size(1), x.size(1),
-                   false);
-  element_wise(InplaceSigmoid{r}, x.size(0) * rw.size(1));
-  gemm_fp16_cublas(kx, kw.data_ptr(), vx, x.size(0), kw.size(1), x.size(1),
-                   false);
-  element_wise(InplaceReLUAndSquare{vx}, x.size(0) * kw.size(1));
-  gemm_fp16_cublas(vx, vw.data_ptr(), x_plus_out.data_ptr(), x.size(0),
-                   vw.size(1), vw.size(0), false);
-  element_wise(InplaceFma{data_ptr<half>(x_plus_out), r, data_ptr<half>(x)},
-               x_plus_out.numel());
-  return xx;
-}
-
-struct FfnOneMix {
-  __device__ __forceinline__ void operator()(int idx) {
-    half k_mix_ = k_mix[idx];
-    half r_mix_ = r_mix[idx];
-    half xx_ = xx[idx];
-    half sx_ = sx[idx];
-    kx[idx] = __hadd(__hmul(xx_, k_mix_),
-                     __hmul(sx_, __hsub(__float2half(1), k_mix_)));
-    rx[idx] = __hadd(__hmul(xx_, r_mix_),
-                     __hmul(sx_, __hsub(__float2half(1), r_mix_)));
-  }
-  half *k_mix;
-  half *r_mix;
-  half *xx;
-  half *sx;
-  half *kx;
-  half *rx;
-};
-
-/*
-  Equivalent Python code:
-
-  xx = F.layer_norm(x, (x.shape[-1],), weight=ln_w, bias=ln_b)
-  kx = xx * k_mix + sx * (1 - k_mix)
-  rx = xx * r_mix + sx * (1 - r_mix)
-
-  r = torch.sigmoid(gemm(rx, rw))
-  vx = torch.square(torch.relu(gemm(kx, kw)))
-  out = r * gemm(vx, vw)
-  return x + out, xx
-*/
-Tensor ffn_one(Tensor x, Tensor sx, Tensor ln_w, Tensor ln_b, Tensor k_mix,
-               Tensor r_mix, Tensor kw, Tensor vw, Tensor rw,
-               /* imm */ Tensor buf,
-               /* out */ Tensor x_plus_out) {
-  Tensor xx = at::layer_norm(x, {x.size(-1)}, ln_w, ln_b);
-  char *buf_ptr = (char *)buf.data_ptr();
-  half *kx = (half *)buf_ptr;
-  half *rx = kx + x.numel();
-  half *vx = rx + x.numel();
-  half *r = vx + x.size(0) * kw.size(1);
-  element_wise(FfnOneMix{data_ptr<half>(k_mix), data_ptr<half>(r_mix),
-                         data_ptr<half>(xx), data_ptr<half>(sx), kx, rx},
-               x.numel());
-  // vector * matrix, so m = 1
-  gemm_fp16_cublas(rx, rw.data_ptr(), r, 1, rw.size(1), rw.size(0), false);
-  element_wise(InplaceSigmoid{r}, rw.size(1));
-  gemm_fp16_cublas(kx, kw.data_ptr(), vx, 1, kw.size(1), kw.size(0), false);
-  element_wise(InplaceReLUAndSquare{vx}, kw.size(1));
-  gemm_fp16_cublas(vx, vw.data_ptr(), x_plus_out.data_ptr(), 1, vw.size(1),
-                   vw.size(0), false);
-  element_wise(InplaceFma{data_ptr<half>(x_plus_out), r, data_ptr<half>(x)},
-               x_plus_out.numel());
-  return xx;
-}
--- a/backend-python/rwkv_pip/cuda/util.h
+++ b/backend-python/rwkv_pip/cuda/util.h
@@ -1,7 +0,0 @@
-#include "ATen/ATen.h"
-#include <cuda_fp16.h>
-
-template <typename T> T *data_ptr(torch::Tensor x) { return x.data_ptr<T>(); }
-template <> inline half *data_ptr(torch::Tensor x) {
-  return reinterpret_cast<half *>(x.data_ptr<at::Half>());
-}
--- a/backend-python/rwkv_pip/model.py
+++ b/backend-python/rwkv_pip/model.py
@@ -92,7 +92,7 @@ if os.environ.get("RWKV_CUDA_ON") == "1":
                    f"{current_path}/cuda/gemm_fp16_cublas.cpp",
                ],
                verbose=True,
-                extra_ldflags=["cublas.lib"],
+                extra_ldflags=["cublas.lib" if os.name == "nt" else ""],
                extra_cuda_cflags=[
                    "--use_fast_math",
                    "-O3",
@@ -220,7 +220,7 @@ class RWKV(MyModule):
        else:
            prxxx = lambda *args, **kwargs: None

-        STRATEGY_REGEX = r"^(?:(?:^|->) *(?:cuda(?::[\d]+)?|cpu|mps) (?:fp(?:16|32)|bf16)(?:i8|i4|i3)?(?: \*[\d]+\+?)? *)+$"
+        STRATEGY_REGEX = r"^(?:(?:^|->) *(?:cuda(?::[\d]+)?|cpu|mps|dml) (?:fp(?:16|32)|bf16)(?:i8|i4|i3)?(?: \*[\d]+\+?)? *)+$"
        if not re.match(STRATEGY_REGEX, strategy):
            raise ValueError(
                "Invalid strategy. Please read https://pypi.org/project/rwkv/"
@@ -372,6 +372,10 @@ class RWKV(MyModule):
                        strategy[n].atype = s[i][1][0]
                        strategy[n].wtype = s[i][1][1]
                        strategy[n].stream = False
+                        if strategy[n].device == "dml":
+                            import torch_directml
+
+                            strategy[n].device = torch_directml.device()
                        if i == stream_i and n >= (plan[i] - stream_count):
                            strategy[n].stream = True
                        break
@@ -577,10 +581,7 @@ class RWKV(MyModule):
                prxxx(f"Converted and saved. Now this will exit.")
                exit(0)

-            if self.version == 5.2:
-                assert (
-                    os.environ["RWKV_CUDA_ON"] == "1"
-                ), "Please Enable Custom CUDA Kernel. Latest RWKV-5 requires os.environ['RWKV_CUDA_ON'] == '1' (will fix soon)"
+            if self.version == 5.2 and os.environ["RWKV_CUDA_ON"] == "1":
                HEAD_SIZE = args.n_att // args.n_head
                if LoadPreCompileLibrary("rwkv5") is True:
                    rwkv5 = torch.ops.rwkv5
@@ -596,6 +597,7 @@ class RWKV(MyModule):
                            "-res-usage",
                            "--use_fast_math",
                            "-O3",
+                            "-Xptxas -O3" if os.name != "nt" else "",
                            "--extra-device-vectorization",
                            f"-D_N_={HEAD_SIZE}",
                        ],
@@ -1362,6 +1364,7 @@ class RWKV(MyModule):

    ########################################################################################################

+    @MyFunction
    def att_seq_v5_2(
        self,
        x,
@@ -1407,29 +1410,29 @@ class RWKV(MyModule):
        gx = xx * g_mix + sx * (1 - g_mix)

        H = t_decay.shape[0]
-        N = x.shape[-1] // H
+        S = x.shape[-1] // H
        T = x.shape[0]

-        r = gemm(rx, rw, output_dtype=torch.float32)
-        k = gemm(kx, kw, output_dtype=torch.float32)
-        v = gemm(vx, vw, output_dtype=torch.float32)
+        r = gemm(rx, rw, output_dtype=torch.float32).view(T, H, S).transpose(0, 1)
+        k = (
+            gemm(kx, kw, output_dtype=torch.float32)
+            .view(T, H, S)
+            .transpose(0, 1)
+            .transpose(-2, -1)
+        )
+        v = gemm(vx, vw, output_dtype=torch.float32).view(T, H, S).transpose(0, 1)
        g = F.silu(gemm(gx, gw))

-        out, s = self.RUN_RWKV_5(
-            1,
-            T,
-            self.args.n_att,
-            H,
-            s.transpose(-1, -2).contiguous(),
-            r,
-            k,
-            v,
-            w=t_decay,
-            u=t_first,
-        )
-        s = s.transpose(-1, -2)
+        out = torch.empty((T, H, S), dtype=r.dtype, device=r.device)
+        for t in range(T):
+            rt = r[:, t : t + 1, :]
+            kt = k[:, :, t : t + 1]
+            vt = v[:, t : t + 1, :]
+            at = gemm(kt, vt)
+            out[t] = (rt @ (t_first * at + s)).squeeze(1)
+            s = at + t_decay * s

-        out = out.reshape(T, H * N)
+        out = out.reshape(T, H * S)
        out = F.group_norm(out, num_groups=H, weight=lx_w, bias=lx_b)
        out = out.to(dtype=x.dtype) * g
        out = gemm(out, ow)
@@ -1542,6 +1545,81 @@ class RWKV(MyModule):
            out = self.mm8_seq(r * y, ow, omx, orx, omy, ory)
            return x + out, xx[-1, :], aa, bb, pp

+        # NOTE: decorate with @MyFunction causes JIT error
+        def cuda_att_seq_v5_2(
+            self,
+            x,
+            sx,
+            s,
+            ln_w,
+            ln_b,
+            lx_w,
+            lx_b,
+            k_mix,
+            v_mix,
+            r_mix,
+            g_mix,
+            t_decay,
+            t_first,
+            kw,
+            vw,
+            rw,
+            gw,
+            ow,
+            kmx,
+            krx,
+            kmy,
+            kry,
+            vmx,
+            vrx,
+            vmy,
+            vry,
+            rmx,
+            rrx,
+            rmy,
+            rry,
+            omx,
+            orx,
+            omy,
+            ory,
+        ):
+            xx = F.layer_norm(x, (x.shape[-1],), weight=ln_w, bias=ln_b)
+            sx = torch.cat((sx.unsqueeze(0), xx[:-1, :]))
+            kx = xx * k_mix + sx * (1 - k_mix)
+            vx = xx * v_mix + sx * (1 - v_mix)
+            rx = xx * r_mix + sx * (1 - r_mix)
+            gx = xx * g_mix + sx * (1 - g_mix)
+
+            H = t_decay.shape[0]
+            N = x.shape[-1] // H
+            T = x.shape[0]
+
+            r = gemm(rx, rw, output_dtype=torch.float32)
+            k = gemm(kx, kw, output_dtype=torch.float32)
+            v = gemm(vx, vw, output_dtype=torch.float32)
+            g = F.silu(gemm(gx, gw))
+
+            out, s = self.RUN_RWKV_5(
+                1,
+                T,
+                self.args.n_att,
+                H,
+                s.transpose(-1, -2).contiguous(),
+                r,
+                k,
+                v,
+                w=t_decay,
+                u=t_first,
+            )
+            s = s.transpose(-1, -2)
+
+            out = out.reshape(T, H * N)
+            out = F.group_norm(out, num_groups=H, weight=lx_w, bias=lx_b)
+            out = out.to(dtype=x.dtype) * g
+            out = gemm(out, ow)
+
+            return x + out, xx[-1, :], s
+
    ########################################################################################################

    def forward(self, tokens, state, full_output=False):
@@ -1621,7 +1699,10 @@ class RWKV(MyModule):
                atype = dd.atype
                wtype = dd.wtype
                if seq_mode:
-                    if "cuda" in str(dev) and os.environ["RWKV_CUDA_ON"] == "1":
+                    cuda_applicable = os.environ[
+                        "RWKV_CUDA_ON"
+                    ] == "1" and "cuda" in str(dev)
+                    if cuda_applicable:
                        ATT = (
                            self.cuda_att_seq
                            if wtype != torch.uint8
@@ -1635,6 +1716,8 @@ class RWKV(MyModule):
                        ATT = self.att_seq_v5_1
                    elif self.version == 5.2:
                        ATT = self.att_seq_v5_2
+                        if cuda_applicable:
+                            ATT = self.cuda_att_seq_v5_2
                    FFN = self.ffn_seq if wtype != torch.uint8 else self.ffn_seq_i8
                else:
                    ATT = self.att_one if wtype != torch.uint8 else self.att_one_i8
--- a/backend-python/rwkv_pip/rwkv5.pyd
+++ b/backend-python/rwkv_pip/rwkv5.pyd
--- a/backend-python/rwkv_pip/wkv_cuda.pyd
+++ b/backend-python/rwkv_pip/wkv_cuda.pyd
--- a/backend-python/utils/rwkv.py
+++ b/backend-python/utils/rwkv.py
@@ -4,7 +4,7 @@ import os
 import pathlib
 import copy
 import re
-from typing import Dict, Iterable, List, Tuple, Union
+from typing import Dict, Iterable, List, Tuple, Union, Type
 from utils.log import quick_log
 from fastapi import HTTPException
 from pydantic import BaseModel, Field
@@ -21,33 +21,21 @@ os.environ["TORCH_EXTENSIONS_DIR"] = f"{pathlib.Path(__file__).parent.parent.res


 class RWKVType(Enum):
+    NoneType = auto()
    Raven = auto()
    World = auto()
    Music = auto()


 class AbstractRWKV(ABC):
-    def __init__(self, model: str, strategy: str, tokens_path: str):
-        rwkv_beta = global_var.get(global_var.Args).rwkv_beta
-
-        # dynamic import to make RWKV_CUDA_ON work
-        if rwkv_beta:
-            from rwkv_pip.beta.model import (
-                RWKV as Model,
-            )
-        else:
-            from rwkv_pip.model import (
-                RWKV as Model,
-            )
-        from rwkv_pip.utils import PIPELINE
-
-        filename, _ = os.path.splitext(os.path.basename(model))
-        self.name = filename
-        self.model = Model(model, strategy)
-        self.pipeline = PIPELINE(self.model, tokens_path)
+    def __init__(self, model, pipeline):
+        self.name = "rwkv"
+        self.model = model
+        self.pipeline = pipeline
        self.model_state = None
        self.model_tokens = []
-        self.rwkv_type: RWKVType = None
+        self.rwkv_type: RWKVType = RWKVType.NoneType
+        self.tokenizer_len = len(model.w["emb.weight"])

        self.max_tokens_per_generation = 500
        self.temperature = 1
@@ -348,8 +336,8 @@ class AbstractRWKV(ABC):


 class TextRWKV(AbstractRWKV):
-    def __init__(self, model: str, strategy: str, tokens_path: str) -> None:
-        super().__init__(model, strategy, tokens_path)
+    def __init__(self, model, pipeline) -> None:
+        super().__init__(model, pipeline)

        self.CHUNK_LEN = 256

@@ -361,16 +349,16 @@ class TextRWKV(AbstractRWKV):
        self.penalty_alpha_frequency = 1

        self.interface = ":"
-        if "world" in self.name.lower():
-            self.rwkv_type = RWKVType.World
-            self.user = "Question"
-            self.bot = "Answer"
-            self.END_OF_LINE = 11
-        else:
+        if self.tokenizer_len < 65536:
            self.rwkv_type = RWKVType.Raven
            self.user = "Bob"
            self.bot = "Alice"
            self.END_OF_LINE = 187
+        else:
+            self.rwkv_type = RWKVType.World
+            self.user = "User"
+            self.bot = "Assistant"
+            self.END_OF_LINE = 11

        self.AVOID_REPEAT_TOKENS = []
        AVOID_REPEAT = "，：？！"
@@ -469,8 +457,8 @@ The following is a coherent verbose detailed conversation between a girl named {


 class MusicRWKV(AbstractRWKV):
-    def __init__(self, model: str, strategy: str, tokens_path: str):
-        super().__init__(model, strategy, tokens_path)
+    def __init__(self, model, pipeline):
+        super().__init__(model, pipeline)

        self.max_tokens_per_generation = 500
        self.temperature = 1
@@ -510,6 +498,52 @@ class MusicRWKV(AbstractRWKV):
        return " " + delta


+def get_tokenizer(tokenizer_len: int):
+    tokenizer_dir = f"{pathlib.Path(__file__).parent.parent.resolve()}/rwkv_pip/"
+    if tokenizer_len < 50277:
+        return tokenizer_dir + "tokenizer-midi.json"
+    elif tokenizer_len < 65536:
+        return tokenizer_dir + "20B_tokenizer.json"
+    else:
+        return "rwkv_vocab_v20230424"
+
+
+def RWKV(model: str, strategy: str, tokenizer: Union[str, None]) -> AbstractRWKV:
+    rwkv_beta = global_var.get(global_var.Args).rwkv_beta
+
+    # dynamic import to make RWKV_CUDA_ON work
+    if rwkv_beta:
+        from rwkv_pip.beta.model import (
+            RWKV as Model,
+        )
+    else:
+        from rwkv_pip.model import (
+            RWKV as Model,
+        )
+    from rwkv_pip.utils import PIPELINE
+
+    filename, _ = os.path.splitext(os.path.basename(model))
+    model = Model(model, strategy)
+    if not tokenizer:
+        tokenizer = get_tokenizer(len(model.w["emb.weight"]))
+    pipeline = PIPELINE(model, tokenizer)
+
+    rwkv_map: dict[str, Type[AbstractRWKV]] = {
+        "20B_tokenizer": TextRWKV,
+        "rwkv_vocab_v20230424": TextRWKV,
+        "tokenizer-midi": MusicRWKV,
+    }
+    tokenizer_name = os.path.splitext(os.path.basename(tokenizer))[0]
+    rwkv: AbstractRWKV
+    if tokenizer_name in rwkv_map:
+        rwkv = rwkv_map[tokenizer_name](model, pipeline)
+    else:
+        rwkv = TextRWKV(model, pipeline)
+    rwkv.name = filename
+
+    return rwkv
+
+
 class ModelConfigBody(BaseModel):
    max_tokens: int = Field(default=None, gt=0, le=102400)
    temperature: float = Field(default=None, ge=0, le=2)
@@ -518,7 +552,7 @@ class ModelConfigBody(BaseModel):
    frequency_penalty: float = Field(default=None, ge=-2, le=2)

    class Config:
-        schema_extra = {
+        json_schema_extra = {
            "example": {
                "max_tokens": 1000,
                "temperature": 1.2,
--- a/finetune/data/sample.jsonl
+++ b/finetune/data/sample.jsonl
@@ -1,7 +1,5 @@
-{"text": "1:This is the first document."}
-{"text": "2:Hello\nWorld"}
-{"text": "3:1+1=2\n1+2=3\n2+2=4"}
-{"text": "4:You will be training the GPT version because it's paralleziable and faster to train."}
-{"text": "5:Read the inference code in src/model.py and try using the final hidden state(.xx .aa .bb)"}
-{"text": "6:You can fine-tune the model with longer ctxLen and it can quickly adapt to longer ctxLens."}
-{"text": "7:Consider RWKV 14B. The state has 200 vectors, that is, 5 vectors for each block: fp16 (xx), fp32 (aa), fp32 (bb), fp32 (pp), fp16 (xx)."}
+{"text": "The following is an epic science fiction masterpiece that is immortalized, with delicate descriptions and grand depictions of interstellar civilization wars.\nChapter 1.\nAs I sit down to write here amidst the shadows of vine-leaves under the blue sky of southern Italy, it comes to me with a certain quality of astonishment that my participation in these amazing adventures of Mr. Cavor was, after all, the outcome of the purest accident. It might have been any one. I fell into these things at a time when I thought myself removed from the slightest possibility of disturbing experiences. I had gone to Lympne because I had imagined it the most uneventful place in the world. “Here, at any rate,” said I, “I shall find peace and a chance to work!”"}
+{"text": "Translate the following into Chinese.\n\nEnglish: What rooms do you have available?\nChinese: 你们有哪些房间可以提供"}
+{"text": "User: Hello.\n\nAssistant: I'm here, meow~.\n\nUser: Can you tell some jokes?\n\nAssistant: Of course, master. What kind of jokes would you like to hear?"}
+{"text": "Instruction: Write a story using the following information\n\nInput: A man named Alex chops a tree down\n\nResponse: Once upon a time, there was a man named Alex who lived in the heart of the forest. He had always been fascinated by trees and spent most of his days exploring the forest and learning about its many wonders. One day, while wandering through the woods, he stumbled upon an old oak tree that stood tall and proud in the middle of a clearing."}
+{"text": "def get_args(args: Union[Sequence[str], None] = None):\n    parser = argparse.ArgumentParser()\n    group = parser.add_argument_group(title=\"server arguments\")\n    group.add_argument(\n        \"--port\",\n        type=int,\n        default=8000,\n        help=\"port to run the server on (default: 8000)\",\n    )\n    group.add_argument(\n        \"--host\",\n        type=str,\n        default=\"127.0.0.1\",\n        help=\"host to run the server on (default: 127.0.0.1)\",\n    )"}
--- a/frontend/src/_locales/ja/main.json
+++ b/frontend/src/_locales/ja/main.json
@@ -100,7 +100,7 @@
  "Model Config Exception": "モデル設定例外",
  "Use Gitee Updates Source": "Gitee更新ソースを使用",
  "Use Custom CUDA kernel to Accelerate": "カスタムCUDAカーネルを使用して加速",
-  "Enabling this option can greatly improve inference speed and save some VRAM, but there may be compatibility issues. If it fails to start, please turn off this option.": "このオプションを有効にすると、推論速度が大幅に向上し、一部のVRAMを節約できますが、互換性の問題が生じる可能性があります。起動に失敗した場合は、このオプションをオフにしてください。",
+  "Enabling this option can greatly improve inference speed and save some VRAM, but there may be compatibility issues (output garbled). If it fails to start, please turn off this option, or try to upgrade your gpu driver.": "このオプションを有効にすると、推論速度が大幅に向上し、一部のVRAMを節約できますが、互換性の問題 (文字化けを出力する) が生じる可能性があります。起動に失敗した場合は、このオプションを無効にするか、GPUドライバーをアップグレードしてみてください。",
  "Supported custom cuda file not found": "対応しているカスタムCUDAファイルが見つかりません",
  "Failed to copy custom cuda file": "カスタムCUDAファイルのコピーに失敗しました",
  "Downloading update, please wait. If it is not completed, please manually download the program from GitHub and replace the original program.": "更新をダウンロード中です、お待ちください。完了しない場合は、GitHubから手動でプログラムをダウンロードし、元のプログラムを置き換えてください。",
@@ -233,7 +233,7 @@
  "Matched CUDA is not installed": "対応するCUDAがインストールされていません",
  "Failed to convert data": "データの変換に失敗しました",
  "Failed to merge model": "モデルのマージに失敗しました",
-  "The data path should be a directory or a file in jsonl format (more formats will be supported in the future).\n\nWhen you provide a directory path, all the txt files within that directory will be automatically converted into training data. This is commonly used for large-scale training in writing, code generation, or knowledge bases.\n\nThe jsonl format file can be referenced at https://github.com/Abel2076/json2binidx_tool/blob/main/sample.jsonl.\nYou can also write it similar to OpenAI's playground format, as shown in https://platform.openai.com/playground/p/default-chat.\nEven for multi-turn conversations, they must be written in a single line using `\\n` to indicate line breaks. If they are different dialogues or topics, they should be written in separate lines.": "データのパスはディレクトリまたはjsonl形式のファイルでなければなりません（将来的にはより多くの形式がサポートされる予定です）。ディレクトリパスを提供した場合、そのディレクトリ内のすべてのtxtファイルが自動的にトレーニングデータに変換されます。これは大規模なライティング、コード生成、または知識ベースのトレーニングで一般的に使用されます。jsonl形式のファイルは、https://github.com/Abel2076/json2binidx_tool/blob/main/sample.jsonl を参照してください。\nhttps://platform.openai.com/playground/p/default-chat のように、OpenAIのプレイグラウンド形式に似た形式で書くこともできます。複数ターンの対話であっても、一行で書く必要があり、行の区切りを示すために`\\n`を使用します。それらが異なる対話やトピックであれば、それらは別々の行に書かれるべきです。",
+  "The data path should be a directory or a file in jsonl format (more formats will be supported in the future).\n\nWhen you provide a directory path, all the txt files within that directory will be automatically converted into training data. This is commonly used for large-scale training in writing, code generation, or knowledge bases.\n\nThe jsonl format file can be referenced at https://github.com/josStorer/RWKV-Runner/blob/master/finetune/data/sample.jsonl.\nYou can also write it similar to OpenAI's playground format, as shown in https://platform.openai.com/playground/p/default-chat.\nEven for multi-turn conversations, they must be written in a single line using `\\n` to indicate line breaks. If they are different dialogues or topics, they should be written in separate lines.": "データのパスはディレクトリまたはjsonl形式のファイルでなければなりません（将来的にはより多くの形式がサポートされる予定です）。ディレクトリパスを提供した場合、そのディレクトリ内のすべてのtxtファイルが自動的にトレーニングデータに変換されます。これは大規模なライティング、コード生成、または知識ベースのトレーニングで一般的に使用されます。jsonl形式のファイルは、https://github.com/josStorer/RWKV-Runner/blob/master/finetune/data/sample.jsonl を参照してください。\nhttps://platform.openai.com/playground/p/default-chat のように、OpenAIのプレイグラウンド形式に似た形式で書くこともできます。複数ターンの対話であっても、一行で書く必要があり、行の区切りを示すために`\\n`を使用します。それらが異なる対話やトピックであれば、それらは別々の行に書かれるべきです。",
  "Size mismatch for blocks. You are attempting to continue training from the LoRA model, but it does not match the base model. Please set LoRA model to None.": "ブロックのサイズが一致しません。LoRAモデルからトレーニングを続けようとしていますが、それはベースモデルと一致しません。LoRAモデルをNoneに設定してください。",
  "Instruction: Write a story using the following information\n\nInput: A man named Alex chops a tree down\n\nResponse:": "Instruction: Write a story using the following information\n\nInput: アレックスという男が木を切り倒す\n\nResponse:",
  "Composition": "作曲",
@@ -254,6 +254,12 @@
  "User Name": "ユーザー名",
  "Assistant Name": "アシスタント名",
  "Insert default system prompt at the beginning": "最初にデフォルトのシステムプロンプトを挿入",
-  "Please Enable Custom CUDA Kernel. Latest RWKV-5 requires os.environ['RWKV_CUDA_ON'] == '1' (will fix soon).": "カスタムCUDAカーネルを有効にしてください。最新のRWKV-5ではos.environ['RWKV_CUDA_ON'] == '1'が必要です（近日中に修正します）。",
-  "Format Content": "内容フォーマットの規格化"
+  "Format Content": "内容フォーマットの規格化",
+  "Add An Attachment (Accepts pdf, txt)": "添付ファイルを追加 (pdf, txtを受け付けます)",
+  "Uploading Attachment": "添付ファイルアップロード中",
+  "Remove Attachment": "添付ファイルを削除",
+  "The content of file": "ファイル",
+  "is as follows. When replying to me, consider the file content and respond accordingly:": "の内容は以下の通りです。私に返信する際は、ファイルの内容を考慮して適切に返信してください:",
+  "What's the file name": "ファイル名は何ですか",
+  "The file name is: ": "ファイル名は次のとおりです: "
 }
--- a/frontend/src/_locales/zh-hans/main.json
+++ b/frontend/src/_locales/zh-hans/main.json
@@ -100,7 +100,7 @@
  "Model Config Exception": "模型配置异常",
  "Use Gitee Updates Source": "使用Gitee更新源",
  "Use Custom CUDA kernel to Accelerate": "使用自定义CUDA算子加速",
-  "Enabling this option can greatly improve inference speed and save some VRAM, but there may be compatibility issues. If it fails to start, please turn off this option.": "开启这个选项能大大提升推理速度并节省显存，但可能存在兼容性问题，如果启动失败，请关闭此选项",
+  "Enabling this option can greatly improve inference speed and save some VRAM, but there may be compatibility issues (output garbled). If it fails to start, please turn off this option, or try to upgrade your gpu driver.": "开启这个选项能大大提升推理速度并节省显存，但可能存在兼容性(回复乱码)问题，如果发生相关问题，请关闭此选项。或更新你的显卡驱动",
  "Supported custom cuda file not found": "没有找到支持的自定义cuda文件",
  "Failed to copy custom cuda file": "自定义cuda文件复制失败",
  "Downloading update, please wait. If it is not completed, please manually download the program from GitHub and replace the original program.": "正在下载更新，请等待。如果一直未完成，请从Github手动下载并覆盖原程序",
@@ -233,7 +233,7 @@
  "Matched CUDA is not installed": "未安装匹配的CUDA",
  "Failed to convert data": "数据转换失败",
  "Failed to merge model": "合并模型失败",
-  "The data path should be a directory or a file in jsonl format (more formats will be supported in the future).\n\nWhen you provide a directory path, all the txt files within that directory will be automatically converted into training data. This is commonly used for large-scale training in writing, code generation, or knowledge bases.\n\nThe jsonl format file can be referenced at https://github.com/Abel2076/json2binidx_tool/blob/main/sample.jsonl.\nYou can also write it similar to OpenAI's playground format, as shown in https://platform.openai.com/playground/p/default-chat.\nEven for multi-turn conversations, they must be written in a single line using `\\n` to indicate line breaks. If they are different dialogues or topics, they should be written in separate lines.": "数据路径必须是一个文件夹，或者jsonl格式文件 (未来会支持更多格式)\n\n当你填写的路径是一个文件夹时，该文件夹内的所有txt文件会被自动转换为训练数据，通常这用于大批量训练写作，代码生成或知识库\n\njsonl文件的格式参考 https://github.com/Abel2076/json2binidx_tool/blob/main/sample.jsonl\n你也可以仿照openai的playground编写，参考 https://platform.openai.com/playground/p/default-chat\n即使是多轮对话也必须写在一行，用`\\n`表示换行，如果是不同对话或主题，则另起一行",
+  "The data path should be a directory or a file in jsonl format (more formats will be supported in the future).\n\nWhen you provide a directory path, all the txt files within that directory will be automatically converted into training data. This is commonly used for large-scale training in writing, code generation, or knowledge bases.\n\nThe jsonl format file can be referenced at https://github.com/josStorer/RWKV-Runner/blob/master/finetune/data/sample.jsonl.\nYou can also write it similar to OpenAI's playground format, as shown in https://platform.openai.com/playground/p/default-chat.\nEven for multi-turn conversations, they must be written in a single line using `\\n` to indicate line breaks. If they are different dialogues or topics, they should be written in separate lines.": "数据路径必须是一个文件夹，或者jsonl格式文件 (未来会支持更多格式)\n\n当你填写的路径是一个文件夹时，该文件夹内的所有txt文件会被自动转换为训练数据，通常这用于大批量训练写作，代码生成或知识库\n\njsonl文件的格式参考 https://github.com/josStorer/RWKV-Runner/blob/master/finetune/data/sample.jsonl 以及 https://zhuanlan.zhihu.com/p/643433851\n你也可以仿照openai的playground编写，参考 https://platform.openai.com/playground/p/default-chat\n即使是多轮对话也必须写在一行，用`\\n`表示换行，如果是不同对话或主题，则另起一行",
  "Size mismatch for blocks. You are attempting to continue training from the LoRA model, but it does not match the base model. Please set LoRA model to None.": "尺寸不匹配块。你正在尝试从LoRA模型继续训练，但该LoRA模型与基底模型不匹配，请将LoRA模型设为空",
  "Instruction: Write a story using the following information\n\nInput: A man named Alex chops a tree down\n\nResponse:": "Instruction: Write a story using the following information\n\nInput: 艾利克斯砍倒了一棵树\n\nResponse:",
  "Composition": "作曲",
@@ -254,6 +254,12 @@
  "User Name": "用户名称",
  "Assistant Name": "AI名称",
  "Insert default system prompt at the beginning": "在开头自动插入默认系统提示",
-  "Please Enable Custom CUDA Kernel. Latest RWKV-5 requires os.environ['RWKV_CUDA_ON'] == '1' (will fix soon).": "请启用自定义CUDA算子。最新的RWKV-5需要os.environ['RWKV_CUDA_ON'] == '1' (未来会修复)",
-  "Format Content": "规范格式"
+  "Format Content": "规范格式",
+  "Add An Attachment (Accepts pdf, txt)": "添加一个附件 (支持pdf, txt)",
+  "Uploading Attachment": "正在上传附件",
+  "Remove Attachment": "移除附件",
+  "The content of file": "文件",
+  "is as follows. When replying to me, consider the file content and respond accordingly:": "内容如下。回复时考虑文件内容并做出相应回复:",
+  "What's the file name": "文件名是什么",
+  "The file name is: ": "文件名是："
 }
--- a/frontend/src/components/RunButton.tsx
+++ b/frontend/src/components/RunButton.tsx
@@ -212,7 +212,6 @@ export const RunButton: FC<{ onClickRun?: MouseEventHandler, iconMode?: boolean
                  'no NVIDIA driver': 'Found no NVIDIA driver, please install the latest driver.',
                  'CUDA out of memory': 'VRAM is not enough, please reduce stored layers or use a lower precision in Configs page.',
                  'Ninja is required to load C++ extensions': 'Failed to enable custom CUDA kernel, ninja is required to load C++ extensions. You may be using the CPU version of PyTorch, please reinstall PyTorch with CUDA. Or if you are using a custom Python interpreter, you must compile the CUDA kernel by yourself or disable Custom CUDA kernel acceleration.',
-                  'Please Enable Custom CUDA Kernel': 'Please Enable Custom CUDA Kernel. Latest RWKV-5 requires os.environ[\'RWKV_CUDA_ON\'] == \'1\' (will fix soon).'
                };
                const matchedError = Object.entries(errorsMap).find(([key, _]) => error.includes(key));
                const message = matchedError ? t(matchedError[1]) : error;
--- a/frontend/src/pages/Chat.tsx
+++ b/frontend/src/pages/Chat.tsx
@@ -10,14 +10,22 @@ import { KebabHorizontalIcon, PencilIcon, SyncIcon, TrashIcon } from '@primer/oc
 import logo from '../assets/images/logo.png';
 import MarkdownRender from '../components/MarkdownRender';
 import { ToolTipButton } from '../components/ToolTipButton';
-import { ArrowCircleUp28Regular, Delete28Regular, RecordStop28Regular, Save28Regular } from '@fluentui/react-icons';
+import {
+  ArrowCircleUp28Regular,
+  ArrowClockwise16Regular,
+  Attach16Regular,
+  Delete28Regular,
+  Dismiss16Regular,
+  RecordStop28Regular,
+  Save28Regular
+} from '@fluentui/react-icons';
 import { CopyButton } from '../components/CopyButton';
 import { ReadButton } from '../components/ReadButton';
 import { toast } from 'react-toastify';
 import { WorkHeader } from '../components/WorkHeader';
 import { DialogButton } from '../components/DialogButton';
-import { OpenFileFolder, OpenSaveFileDialog } from '../../wailsjs/go/backend_golang/App';
-import { toastWithButton } from '../utils';
+import { OpenFileFolder, OpenOpenFileDialog, OpenSaveFileDialog } from '../../wailsjs/go/backend_golang/App';
+import { absPathAsset, bytesToReadable, toastWithButton } from '../utils';
 import { PresetsButton } from './PresetsManager/PresetsButton';
 import { useMediaQuery } from 'usehooks-ts';

@@ -57,7 +65,7 @@ export type ConversationMessage = {
  content: string;
 }

-let chatSseController: AbortController | null = null;
+let chatSseControllers: { [id: string]: AbortController } = {};

 const MoreUtilsButton: FC<{ uuid: string, setEditing: (editing: boolean) => void }> = observer(({
  uuid,
@@ -114,6 +122,13 @@ const ChatMessageItem: FC<{
    }
  };

+  let avatarImg: string | undefined;
+  if (commonStore.activePreset && messageItem.sender === botName) {
+    avatarImg = absPathAsset(commonStore.activePreset.avatarImg);
+  } else if (messageItem.avatarImg) {
+    avatarImg = messageItem.avatarImg;
+  }
+
  return <div
    className={classnames(
      'flex gap-2 mb-2 overflow-hidden',
@@ -131,7 +146,7 @@ const ChatMessageItem: FC<{
    <Avatar
      color={messageItem.color}
      name={messageItem.sender}
-      image={(commonStore.activePreset && messageItem.sender === botName) ? { src: commonStore.activePreset.avatarImg } : messageItem.avatarImg ? { src: messageItem.avatarImg } : undefined}
+      image={avatarImg ? { src: avatarImg } : undefined}
    />
    <div
      className={classnames(
@@ -149,6 +164,10 @@ const ChatMessageItem: FC<{
          value={messageItem.content}
          onChange={(e) => {
            messageItem.content = e.target.value;
+            commonStore.conversation[uuid].type = MessageType.Normal;
+            commonStore.conversation[uuid].done = true;
+            commonStore.setConversation(commonStore.conversation);
+            commonStore.setConversationOrder([...commonStore.conversationOrder]);
          }}
          onBlur={() => {
            setEditingInner(false);
@@ -166,6 +185,10 @@ const ChatMessageItem: FC<{
          messageItem.sender === botName && uuid !== welcomeUuid &&
          <ToolTipButton desc={t('Retry')} size="small" appearance="subtle"
            icon={<SyncIcon />} onClick={() => {
+            if (uuid in chatSseControllers) {
+              chatSseControllers[uuid].abort();
+              delete chatSseControllers[uuid];
+            }
            onSubmit(null, uuid, null, uuid, false);
          }} />
        }
@@ -187,15 +210,7 @@ const ChatPanel: FC = observer(() => {
  const currentConfig = commonStore.getCurrentModelConfig();
  const apiParams = currentConfig.apiParameters;
  const port = apiParams.apiPort;
-
-  let lastMessageId: string;
-  let generating: boolean = false;
-  if (commonStore.conversationOrder.length > 0) {
-    lastMessageId = commonStore.conversationOrder[commonStore.conversationOrder.length - 1];
-    const lastMessage = commonStore.conversation[lastMessageId];
-    if (lastMessage.sender === botName)
-      generating = !lastMessage.done;
-  }
+  const generating: boolean = Object.keys(chatSseControllers).length > 0;

  useEffect(() => {
    if (inputRef.current)
@@ -267,6 +282,16 @@ const ChatPanel: FC = observer(() => {
    let targetRange = commonStore.conversationOrder.slice(startIndex, endIndex);

    const messages: ConversationMessage[] = [];
+    if (commonStore.attachmentContent) {
+      messages.push({
+        role: 'user',
+        content: t('The content of file') + ` "${commonStore.attachmentName}" `
+          + t('is as follows. When replying to me, consider the file content and respond accordingly:')
+          + '\n\n' + commonStore.attachmentContent
+      });
+      messages.push({ role: 'user', content: t('What\'s the file name') });
+      messages.push({ role: 'assistant', content: t('The file name is: ') + commonStore.attachmentName });
+    }
    targetRange.forEach((uuid, index) => {
      if (uuid === welcomeUuid)
        return;
@@ -296,7 +321,8 @@ const ChatPanel: FC = observer(() => {
    commonStore.setConversationOrder(commonStore.conversationOrder);
    setTimeout(scrollToBottom);
    let answer = '';
-    chatSseController = new AbortController();
+    const chatSseController = new AbortController();
+    chatSseControllers[answerId] = chatSseController;
    fetchEventSource( // https://api.openai.com/v1/chat/completions || http://127.0.0.1:${port}/chat/completions
      commonStore.settings.apiUrl ?
        commonStore.settings.apiUrl + '/v1/chat/completions' :
@@ -350,6 +376,8 @@ const ChatPanel: FC = observer(() => {
          }
        },
        onclose() {
+          if (answerId! in chatSseControllers)
+            delete chatSseControllers[answerId!];
          console.log('Connection closed');
        },
        onerror(err) {
@@ -380,33 +408,123 @@ const ChatPanel: FC = observer(() => {
          size={mq ? 'large' : 'small'} shape="circular" appearance="subtle" title={t('Clear')}
          contentText={t('Are you sure you want to clear the conversation? It cannot be undone.')}
          onConfirm={() => {
-            if (generating)
-              chatSseController?.abort();
+            if (generating) {
+              for (const id in chatSseControllers) {
+                chatSseControllers[id].abort();
+              }
+              chatSseControllers = {};
+            }
            commonStore.setConversation({});
            commonStore.setConversationOrder([]);
          }} />
-        <Textarea
-          ref={inputRef}
-          style={{ minWidth: 0 }}
-          className="grow"
-          resize="vertical"
-          placeholder={t('Type your message here')!}
-          value={commonStore.currentInput}
-          onChange={(e) => commonStore.setCurrentInput(e.target.value)}
-          onKeyDown={handleKeyDownOrClick}
-        />
+        <div className="relative flex grow">
+          <Textarea
+            ref={inputRef}
+            style={{ minWidth: 0 }}
+            className="grow"
+            resize="vertical"
+            placeholder={t('Type your message here')!}
+            value={commonStore.currentInput}
+            onChange={(e) => commonStore.setCurrentInput(e.target.value)}
+            onKeyDown={handleKeyDownOrClick}
+          />
+          <div className="absolute right-2 bottom-2">
+            {!commonStore.attachmentContent ?
+              <ToolTipButton
+                desc={commonStore.attachmentUploading ?
+                  t('Uploading Attachment') :
+                  t('Add An Attachment (Accepts pdf, txt)')}
+                icon={commonStore.attachmentUploading ?
+                  <ArrowClockwise16Regular className="animate-spin" />
+                  : <Attach16Regular />}
+                size="small" shape="circular" appearance="secondary"
+                onClick={() => {
+                  if (commonStore.status.status === ModelStatus.Offline && !commonStore.settings.apiUrl) {
+                    toast(t('Please click the button in the top right corner to start the model'), { type: 'warning' });
+                    return;
+                  }
+
+                  if (commonStore.attachmentUploading)
+                    return;
+
+                  OpenOpenFileDialog('*.txt;*.pdf').then(async filePath => {
+                    if (!filePath)
+                      return;
+
+                    commonStore.setAttachmentUploading(true);
+
+                    // Both are slow. Communication between frontend and backend is slow. Use AssetServer Handler to read the file.
+                    // const blob = new Blob([atob(info.content as unknown as string)]); // await fetch(`data:application/octet-stream;base64,${info.content}`).then(r => r.blob());
+                    const blob = await fetch(absPathAsset(filePath)).then(r => r.blob());
+                    const attachmentName = filePath.split(/[\\/]/).pop();
+                    const urlPath = `/file-to-text?file_name=${attachmentName}`;
+                    const bodyForm = new FormData();
+                    bodyForm.append('file_data', blob, attachmentName);
+                    fetch(commonStore.settings.apiUrl ?
+                      commonStore.settings.apiUrl + urlPath :
+                      `http://127.0.0.1:${port}${urlPath}`, {
+                      method: 'POST',
+                      body: bodyForm
+                    }).then(async r => {
+                        if (r.status === 200) {
+                          const pages = (await r.json()).pages as any[];
+                          let attachmentContent: string;
+                          if (pages.length === 1)
+                            attachmentContent = pages[0].page_content;
+                          else
+                            attachmentContent = pages.map((p, i) => `Page ${i + 1}:\n${p.page_content}`).join('\n\n');
+                          commonStore.setAttachmentName(attachmentName!);
+                          commonStore.setAttachmentSize(blob.size);
+                          commonStore.setAttachmentContent(attachmentContent);
+                        } else {
+                          toast(r.statusText + '\n' + (await r.text()), {
+                            type: 'error'
+                          });
+                        }
+                        commonStore.setAttachmentUploading(false);
+                      }
+                    ).catch(e => {
+                      commonStore.setAttachmentUploading(false);
+                      toast(t('Error') + ' - ' + (e.message || e), { type: 'error', autoClose: 2500 });
+                    });
+                  }).catch(e => {
+                    toast(t('Error') + ' - ' + (e.message || e), { type: 'error', autoClose: 2500 });
+                  });
+                }}
+              /> :
+              <div>
+                <ToolTipButton
+                  text={
+                    commonStore.attachmentName.replace(
+                      new RegExp('(^[^\\.]{5})[^\\.]+'), '$1...')
+                  }
+                  desc={`${commonStore.attachmentName} (${bytesToReadable(commonStore.attachmentSize)})`}
+                  size="small" shape="circular" appearance="secondary" />
+                <ToolTipButton desc={t('Remove Attachment')}
+                  icon={<Dismiss16Regular />}
+                  size="small" shape="circular" appearance="subtle"
+                  onClick={() => {
+                    commonStore.setAttachmentName('');
+                    commonStore.setAttachmentSize(0);
+                    commonStore.setAttachmentContent('');
+                  }} />
+              </div>
+            }
+          </div>
+        </div>
        <ToolTipButton desc={generating ? t('Stop') : t('Send')}
          icon={generating ? <RecordStop28Regular /> : <ArrowCircleUp28Regular />}
          size={mq ? 'large' : 'small'} shape="circular" appearance="subtle"
          onClick={(e) => {
            if (generating) {
-              chatSseController?.abort();
-              if (lastMessageId) {
-                commonStore.conversation[lastMessageId].type = MessageType.Error;
-                commonStore.conversation[lastMessageId].done = true;
-                commonStore.setConversation(commonStore.conversation);
-                commonStore.setConversationOrder([...commonStore.conversationOrder]);
+              for (const id in chatSseControllers) {
+                chatSseControllers[id].abort();
+                commonStore.conversation[id].type = MessageType.Error;
+                commonStore.conversation[id].done = true;
              }
+              chatSseControllers = {};
+              commonStore.setConversation(commonStore.conversation);
+              commonStore.setConversationOrder([...commonStore.conversationOrder]);
            } else {
              handleKeyDownOrClick(e);
            }
@@ -417,8 +535,8 @@ const ChatPanel: FC = observer(() => {
          onClick={() => {
            let savedContent: string = '';
            const isWorldModel = commonStore.getCurrentModelConfig().modelParameters.modelName.toLowerCase().includes('world');
-            const user = isWorldModel ? 'Question' : 'Bob';
-            const bot = isWorldModel ? 'Answer' : 'Alice';
+            const user = isWorldModel ? 'User' : 'Bob';
+            const bot = isWorldModel ? 'Assistant' : 'Alice';
            commonStore.conversationOrder.forEach((uuid) => {
              if (uuid === welcomeUuid)
                return;
--- a/frontend/src/pages/Composition.tsx
+++ b/frontend/src/pages/Composition.tsx
@@ -319,7 +319,7 @@ const CompositionPanel: FC = observer(() => {
                    toastWithButton(t('File Saved'), t('Open'), () => {
                      OpenFileFolder(path, false);
                    });
-                }).catch((e: any) => {
+                }).catch((e) => {
                  toast(t('Error') + ' - ' + (e.message || e), { type: 'error', autoClose: 2500 });
                });
              } else {
--- a/frontend/src/pages/Configs.tsx
+++ b/frontend/src/pages/Configs.tsx
@@ -423,7 +423,7 @@ export const Configs: FC = observer(() => {
                {
                  (selectedConfig.modelParameters.device.includes('CUDA') || selectedConfig.modelParameters.device === 'Custom') &&
                  <Labeled label={t('Use Custom CUDA kernel to Accelerate')}
-                    desc={t('Enabling this option can greatly improve inference speed and save some VRAM, but there may be compatibility issues. If it fails to start, please turn off this option.')}
+                    desc={t('Enabling this option can greatly improve inference speed and save some VRAM, but there may be compatibility issues (output garbled). If it fails to start, please turn off this option, or try to upgrade your gpu driver.')}
                    content={
                      <Switch checked={selectedConfig.modelParameters.useCustomCuda}
                        onChange={(e, data) => {
--- a/frontend/src/pages/PresetsManager/PresetsButton.tsx
+++ b/frontend/src/pages/PresetsManager/PresetsButton.tsx
@@ -36,6 +36,7 @@ import { ClipboardGetText, ClipboardSetText } from '../../../wailsjs/runtime';
 import { toast } from 'react-toastify';
 import { CustomToastContainer } from '../../components/CustomToastContainer';
 import { v4 as uuid } from 'uuid';
+import { absPathAsset } from '../../utils';

 export type PresetType = 'chat' | 'completion' | 'chatInCompletion'

@@ -124,7 +125,7 @@ export const PresetCard: FC<{
  const { t } = useTranslation();

  return <PresetCardFrame onClick={onClick}>
-    <img src={avatarImg} className="rounded-xl select-none ml-auto mr-auto h-28" />
+    <img src={absPathAsset(avatarImg)} className="rounded-xl select-none ml-auto mr-auto h-28" />
    <Text size={400}>{name}</Text>
    <Text size={200} style={{
      overflow: 'hidden', textOverflow: 'ellipsis',
@@ -167,8 +168,14 @@ export const ChatPresetEditor: FC<{
  const importPreset = () => {
    ClipboardGetText().then((text) => {
      try {
+        if (!text.trim().startsWith('{'))
+          text = new TextDecoder().decode(
+            new Uint8Array(atob(text)
+            .split('')
+            .map((c) => c.charCodeAt(0))));
        const preset = JSON.parse(text);
        setEditingPreset(preset);
+        setEditingMessages(false);
        toast(t('Imported successfully'), {
          type: 'success',
          autoClose: 1000
@@ -242,7 +249,7 @@ export const ChatPresetEditor: FC<{
              <Button appearance="subtle" icon={<Dismiss20Regular />} />
            </DialogTrigger>
          </div>
-          <img src={editingPreset.avatarImg} className="rounded-xl select-none ml-auto mr-auto h-28" />
+          <img src={absPathAsset(editingPreset.avatarImg)} className="rounded-xl select-none ml-auto mr-auto h-28" />
          <Labeled flex breakline label={t('Name')}
            content={
              <div className="flex gap-2">
--- a/frontend/src/pages/Train.tsx
+++ b/frontend/src/pages/Train.tsx
@@ -414,7 +414,7 @@ const LoraFinetune: FC = observer(() => {
                  contentText={t('The data path should be a directory or a file in jsonl format (more formats will be supported in the future).\n\n' +
                    'When you provide a directory path, all the txt files within that directory will be automatically converted into training data. ' +
                    'This is commonly used for large-scale training in writing, code generation, or knowledge bases.\n\n' +
-                    'The jsonl format file can be referenced at https://github.com/Abel2076/json2binidx_tool/blob/main/sample.jsonl.\n' +
+                    'The jsonl format file can be referenced at https://github.com/josStorer/RWKV-Runner/blob/master/finetune/data/sample.jsonl.\n' +
                    'You can also write it similar to OpenAI\'s playground format, as shown in https://platform.openai.com/playground/p/default-chat.\n' +
                    'Even for multi-turn conversations, they must be written in a single line using `\\n` to indicate line breaks. ' +
                    'If they are different dialogues or topics, they should be written in separate lines.')} />
--- a/frontend/src/stores/commonStore.ts
+++ b/frontend/src/stores/commonStore.ts
@@ -54,6 +54,10 @@ class CommonStore {
  conversation: Conversation = {};
  conversationOrder: string[] = [];
  activePreset: Preset | null = null;
+  attachmentUploading: boolean = false;
+  attachmentName: string = '';
+  attachmentSize: number = 0;
+  attachmentContent: string = '';
  // completion
  completionPreset: CompletionPreset | null = null;
  completionGenerating: boolean = false;
@@ -325,6 +329,22 @@ class CommonStore {
  setLoraModels(value: string[]) {
    this.loraModels = value;
  }
+
+  setAttachmentUploading(value: boolean) {
+    this.attachmentUploading = value;
+  }
+
+  setAttachmentName(value: string) {
+    this.attachmentName = value;
+  }
+
+  setAttachmentSize(value: number) {
+    this.attachmentSize = value;
+  }
+
+  setAttachmentContent(value: string) {
+    this.attachmentContent = value;
+  }
 }

 export default new CommonStore();
--- a/frontend/src/utils/index.tsx
+++ b/frontend/src/utils/index.tsx
@@ -282,6 +282,21 @@ export function bytesToKb(size: number) {
  return (size / 1024).toFixed(2);
 }

+export function bytesToReadable(size: number) {
+  if (size < 1024) return size + ' B';
+  else if (size < 1024 * 1024) return bytesToKb(size) + ' KB';
+  else if (size < 1024 * 1024 * 1024) return bytesToMb(size) + ' MB';
+  else return bytesToGb(size) + ' GB';
+}
+
+export function absPathAsset(path: string) {
+  if ((path.length > 0 && path[0] === '/') ||
+    (path.length > 1 && path[1] === ':')) {
+    return '=>' + path;
+  }
+  return path;
+}
+
 export async function checkUpdate(notifyEvenLatest: boolean = false) {
  fetch(!commonStore.settings.giteeUpdatesSource ?
    'https://api.github.com/repos/josstorer/RWKV-Runner/releases/latest' :
--- a/frontend/wailsjs/go/backend_golang/App.d.ts
+++ b/frontend/wailsjs/go/backend_golang/App.d.ts
@@ -34,6 +34,8 @@ export function MergeLora(arg1:string,arg2:boolean,arg3:number,arg4:string,arg5:

 export function OpenFileFolder(arg1:string,arg2:boolean):Promise<void>;

+export function OpenOpenFileDialog(arg1:string):Promise<string>;
+
 export function OpenSaveFileDialog(arg1:string,arg2:string,arg3:string):Promise<string>;

 export function OpenSaveFileDialogBytes(arg1:string,arg2:string,arg3:Array<number>):Promise<string>;
--- a/frontend/wailsjs/go/backend_golang/App.js
+++ b/frontend/wailsjs/go/backend_golang/App.js
@@ -66,6 +66,10 @@ export function OpenFileFolder(arg1, arg2) {
  return window['go']['backend_golang']['App']['OpenFileFolder'](arg1, arg2);
 }

+export function OpenOpenFileDialog(arg1) {
+  return window['go']['backend_golang']['App']['OpenOpenFileDialog'](arg1);
+}
+
 export function OpenSaveFileDialog(arg1, arg2, arg3) {
  return window['go']['backend_golang']['App']['OpenSaveFileDialog'](arg1, arg2, arg3);
 }
--- a/main.go
+++ b/main.go
@@ -27,6 +27,7 @@ func NewFileLoader() *FileLoader {
 func (h *FileLoader) ServeHTTP(res http.ResponseWriter, req *http.Request) {
 	var err error
 	requestedFilename := strings.TrimPrefix(req.URL.Path, "/")
+	requestedFilename = strings.TrimPrefix(requestedFilename, "=>") // absolute path
 	println("Requesting file:", requestedFilename)
 	fileData, err := os.ReadFile(requestedFilename)
 	if err != nil {
@@ -43,7 +44,7 @@ var assets embed.FS
 //go:embed all:py310/Lib/site-packages/cyac
 var cyac embed.FS

-//go:embed all:py310/Lib/site-packages/cyac-1.7.dist-info
+//go:embed all:py310/Lib/site-packages/cyac-1.9.dist-info
 var cyacInfo embed.FS

 //go:embed backend-python
@@ -66,6 +67,7 @@ var components embed.FS

 func main() {
 	if buildInfo, ok := debug.ReadBuildInfo(); !ok || strings.Contains(buildInfo.String(), "-ldflags") {
+		os.RemoveAll("./py310/Lib/site-packages/cyac-1.7.dist-info")
 		backend.CopyEmbed(cyac)
 		backend.CopyEmbed(cyacInfo)
 		backend.CopyEmbed(py)
@@ -93,11 +95,11 @@ func main() {

 	// Create application with options
 	err = wails.Run(&options.App{
-		Title:     "RWKV-Runner",
-		Width:     1024,
-		Height:    680,
-		MinWidth:  375,
-		MinHeight: 640,
+		Title:                    "RWKV-Runner",
+		Width:                    1024,
+		Height:                   680,
+		MinWidth:                 375,
+		MinHeight:                640,
 		EnableDefaultContextMenu: true,
 		Windows: &windows.Options{
 			ZoomFactor:           zoomFactor,
--- a/manifest.json
+++ b/manifest.json
@@ -1,5 +1,5 @@
 {
-  "version": "1.4.7",
+  "version": "1.4.8",
  "introduction": {
    "en": "RWKV is an open-source, commercially usable large language model with high flexibility and great potential for development.\n### About This Tool\nThis tool aims to lower the barrier of entry for using large language models, making it accessible to everyone. It provides fully automated dependency and model management. You simply need to click and run, following the instructions, to deploy a local large language model. The tool itself is very compact and only requires a single executable file for one-click deployment.\nAdditionally, this tool offers an interface that is fully compatible with the OpenAI API. This means you can use any ChatGPT client as a client for RWKV, enabling capability expansion beyond just chat functionality.\n### Preset Configuration Rules at the Bottom\nThis tool comes with a series of preset configurations to reduce complexity. The naming rules for each configuration represent the following in order: device - required VRAM/memory - model size - model language.\nFor example, \"GPU-8G-3B-EN\" indicates that this configuration is for a graphics card with 8GB of VRAM, a model size of 3 billion parameters, and it uses an English language model.\nLarger model sizes have higher performance and VRAM requirements. Among configurations with the same model size, those with higher VRAM usage will have faster runtime.\nFor example, if you have 12GB of VRAM but running the \"GPU-12G-7B-EN\" configuration is slow, you can downgrade to \"GPU-8G-3B-EN\" for a significant speed improvement.\n### About RWKV\nRWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the \"GPT\" mode to quickly compute the hidden state for the \"RNN\" mode.<br/>So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, \"infinite\" ctx_len, and free sentence embedding (using the final hidden state).",
    "zh": "RWKV是一个开源且允许商用的大语言模型，灵活性很高且极具发展潜力。\n### 关于本工具\n本工具旨在降低大语言模型的使用门槛，做到人人可用，本工具提供了全自动化的依赖和模型管理，你只需要直接点击运行，跟随引导，即可完成本地大语言模型的部署，工具本身体积极小，只需要一个exe即可完成一键部署。\n此外，本工具提供了与OpenAI API完全兼容的接口，这意味着你可以把任意ChatGPT客户端用作RWKV的客户端，实现能力拓展，而不局限于聊天。\n### 底部的预设配置规则\n本工具内置了一系列预设配置，以降低使用难度，每个配置名的规则，依次代表着：设备-所需显存/内存-模型规模-模型语言。\n例如，GPU-8G-3B-CN，表示该配置用于显卡，需要8G显存，模型规模为30亿参数，使用的是中文模型。\n模型规模越大，性能要求越高，显存要求也越高，而同样模型规模的配置中，显存占用越高的，运行速度越快。\n例如当你有12G显存，但运行GPU-12G-7B-CN配置速度比较慢，可降级成GPU-8G-3B-CN，将会大幅提速。\n### 关于RWKV\nRWKV是具有Transformer级别LLM性能的RNN，也可以像GPT Transformer一样直接进行训练（可并行化）。而且它是100% attention-free的。你只需在位置t处获得隐藏状态即可计算位置t + 1处的状态。你可以使用“GPT”模式快速计算用于“RNN”模式的隐藏状态。\n因此，它将RNN和Transformer的优点结合起来 - 高性能、快速推理、节省显存、快速训练、“无限”上下文长度以及免费的语句嵌入（使用最终隐藏状态）。"
@@ -15,6 +15,19 @@
    }
  ],
  "models": [
+    {
+      "name": "RWKV-5-World-1B5-v2-20231025-ctx4096.pth",
+      "desc": {
+        "en": "RWKV-5 Global Languages 1.5B v2",
+        "zh": "RWKV-5 全球语言 1.5B v2",
+        "ja": "RWKV-5 グローバル言語 1.5B v2"
+      },
+      "size": 3155590194,
+      "SHA256": "5a89f56be7f82ab9dd0835af9a6838f788477471616c02f7b041e3aea0c57435",
+      "lastUpdated": "2023-10-26T05:49:30",
+      "url": "https://huggingface.co/BlinkDL/rwkv-5-world/blob/main/RWKV-5-World-1B5-v2-20231025-ctx4096.pth",
+      "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-5-world/resolve/main/RWKV-5-World-1B5-v2-20231025-ctx4096.pth"
+    },
    {
      "name": "RWKV-4-World-CHNtuned-0.1B-v1-20230617-ctx4096.pth",
      "desc": {
@@ -507,8 +520,8 @@
    {
      "name": "RWKV-4-Raven-1B5-v11-Eng99%-Other1%-20230425-ctx4096.pth",
      "desc": {
-        "en": "English 1.5B v11",
-        "zh": "英文 1.5B v11"
+        "en": "English 1.5B v11 (Old Model)",
+        "zh": "英文 1.5B v11 (旧模型)"
      },
      "size": 3030279730,
      "SHA256": "4ac715aecc5b1c90e8e37eebb8163392699066ec23b18144416e91cb4e78675a",
@@ -520,8 +533,8 @@
    {
      "name": "RWKV-4-Raven-1B5-v12-Eng98%-Other2%-20230520-ctx4096.pth",
      "desc": {
-        "en": "English 1B5 v12",
-        "zh": "英文 1B5 v12"
+        "en": "English 1B5 v12 (Old Model)",
+        "zh": "英文 1B5 v12 (旧模型)"
      },
      "size": 3030279730,
      "SHA256": "6bbbffb3ee2372dfa9ef49c599e9a2bc0a01b94b6a264ba9bf5bd524fc38f723",
@@ -532,8 +545,8 @@
    {
      "name": "RWKV-4-Raven-3B-v11-Eng99%-Other1%-20230425-ctx4096.pth",
      "desc": {
-        "en": "English 3B v11",
-        "zh": "英文 3B v11"
+        "en": "English 3B v11 (Old Model)",
+        "zh": "英文 3B v11 (旧模型)"
      },
      "size": 5969345074,
      "SHA256": "982ad3d794efe58992db23c6d694c57a9e62d54718264ec6d6acfae5eb0eea12",
@@ -545,8 +558,8 @@
    {
      "name": "RWKV-4-Raven-3B-v12-Eng98%-Other2%-20230520-ctx4096.pth",
      "desc": {
-        "en": "English 3B v12",
-        "zh": "英文 3B v12"
+        "en": "English 3B v12 (Old Model)",
+        "zh": "英文 3B v12 (旧模型)"
      },
      "size": 5969345074,
      "SHA256": "1eea1845acfe9729dfdaec66a8d1aeb91a1287d94bebbca5529c13c050540b33",
@@ -557,8 +570,8 @@
    {
      "name": "RWKV-4-Raven-3B-v11-Eng49%-Chn49%-Jpn1%-Other1%-20230429-ctx4096.pth",
      "desc": {
-        "en": "Chinese 3B v11",
-        "zh": "中文 3B v11"
+        "en": "Chinese 3B v11 (Old Model)",
+        "zh": "中文 3B v11 (旧模型)"
      },
      "size": 5969345074,
      "SHA256": "af12300d9875e0e166c23d6e9b20928db435073060bf1d36f874060de92ada98",
@@ -570,8 +583,8 @@
    {
      "name": "RWKV-4-Raven-3B-v12-Eng49%-Chn49%-Jpn1%-Other1%-20230527-ctx4096.pth",
      "desc": {
-        "en": "Chinese 3B v12",
-        "zh": "中文 3B v12"
+        "en": "Chinese 3B v12 (Old Model)",
+        "zh": "中文 3B v12 (旧模型)"
      },
      "size": 5969345330,
      "SHA256": "c0abb4b745ba3523b9d8b3e1293110867ee55b1ef3dc8c122212f78396755721",
@@ -582,8 +595,8 @@
    {
      "name": "RWKV-4-Raven-7B-v11x-Eng99%-Other1%-20230429-ctx8192.pth",
      "desc": {
-        "en": "English 7B v11x",
-        "zh": "英文 7B v11x"
+        "en": "English 7B v11x (Old Model)",
+        "zh": "英文 7B v11x (旧模型)"
      },
      "size": 14785389874,
      "SHA256": "f00d5c75b453f2b20ad875fb5a324564c34024eea25a015f5eb441e4f364c3fe",
@@ -595,8 +608,8 @@
    {
      "name": "RWKV-4-Raven-7B-v12-Eng98%-Other2%-20230521-ctx8192.pth",
      "desc": {
-        "en": "English 7B v12",
-        "zh": "英文 7B v12"
+        "en": "English 7B v12 (Old Model)",
+        "zh": "英文 7B v12 (旧模型)"
      },
      "size": 14785389618,
      "SHA256": "5a725eaeb9e09b724de6c97e6845dd0283097c7920acd05b46852ab7afa9ec32",
@@ -607,8 +620,8 @@
    {
      "name": "RWKV-4-Raven-7B-v10x-Eng49%-Chn50%-Other1%-20230423-ctx4096.pth",
      "desc": {
-        "en": "Chinese 7B v10x",
-        "zh": "中文 7B v10x"
+        "en": "Chinese 7B v10x (Old Model)",
+        "zh": "中文 7B v10x (旧模型)"
      },
      "size": 14785389874,
      "SHA256": "7aaf40bb3d440a949db3a146b0a5bbb3e925942b496775b51f5630a582fc236d",
@@ -620,8 +633,8 @@
    {
      "name": "RWKV-4-Raven-7B-v11-Eng49%-Chn49%-Jpn1%-Other1%-20230430-ctx8192.pth",
      "desc": {
-        "en": "Chinese 7B v11",
-        "zh": "中文 7B v11"
+        "en": "Chinese 7B v11 (Old Model)",
+        "zh": "中文 7B v11 (旧模型)"
      },
      "size": 14785389874,
      "SHA256": "9e67a74964abcb4463711e447ddf47735561d7b40592d2d02b29d2e796a4fd14",
@@ -633,8 +646,8 @@
    {
      "name": "RWKV-4-Raven-7B-v12-Eng49%-Chn49%-Jpn1%-Other1%-20230530-ctx8192.pth",
      "desc": {
-        "en": "Chinese 7B v12",
-        "zh": "中文 7B v12"
+        "en": "Chinese 7B v12 (Old Model)",
+        "zh": "中文 7B v12 (旧模型)"
      },
      "size": 14785389874,
      "SHA256": "6d4a089ff36d5d9d96b669d425fc5e4e3959cab426535b52e2364df08f58b407",
@@ -645,8 +658,8 @@
    {
      "name": "RWKV-4-Raven-14B-v11x-Eng99%-Other1%-20230501-ctx8192.pth",
      "desc": {
-        "en": "English 14B v11x",
-        "zh": "英文 14B v11x"
+        "en": "English 14B v11x (Old Model)",
+        "zh": "英文 14B v11x (旧模型)"
      },
      "size": 28297309490,
      "SHA256": "c4bc72406c3c62613e8e2592e8d07ac045f8a88381c728f8eb60af890e299f4d",
@@ -658,8 +671,8 @@
    {
      "name": "RWKV-4-Raven-14B-v12-Eng98%-Other2%-20230523-ctx8192.pth",
      "desc": {
-        "en": "English 14B v12",
-        "zh": "英文 14B v12"
+        "en": "English 14B v12 (Old Model)",
+        "zh": "英文 14B v12 (旧模型)"
      },
      "size": 28297309490,
      "SHA256": "1193b5a9ceab572e4dbb9ed1d798eab7bf4793d18904d08bd4bf183579338ae7",
@@ -692,6 +705,32 @@
      "lastUpdated": "2023-07-17T15:02:08",
      "url": "https://huggingface.co/BlinkDL/rwkv-4-music/blob/main/RWKV-4-MIDI-560M-v1-20230717-ctx4096.pth",
      "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-music/resolve/main/RWKV-4-MIDI-560M-v1-20230717-ctx4096.pth"
+    },
+    {
+      "name": "RWKV-5-MIDI-120M-v1-20230728-ctx4096.pth",
+      "desc": {
+        "en": "RWKV-5 Music 120M v1",
+        "zh": "RWKV-5 作曲 120M v1",
+        "ja": "RWKV-5 作曲 120M v1"
+      },
+      "size": 245070513,
+      "SHA256": "c43d4a2ee7a71a331d05d6cd818dd75f7c48c716e4b98c58e4d27231614b0144",
+      "lastUpdated": "2023-07-29T02:17:27",
+      "url": "https://huggingface.co/BlinkDL/rwkv-5-music/blob/main/RWKV-5-MIDI-120M-v1-20230728-ctx4096.pth",
+      "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-5-music/resolve/main/RWKV-5-MIDI-120M-v1-20230728-ctx4096.pth"
+    },
+    {
+      "name": "RWKV-5-MIDI-560M-v1-20230902-ctx4096.pth",
+      "desc": {
+        "en": "RWKV-5 Music 560M v1",
+        "zh": "RWKV-5 作曲 560M v1",
+        "ja": "RWKV-5 作曲 560M v1"
+      },
+      "size": 1179631346,
+      "SHA256": "cb4f2fd8956ca8496d6b2e33bff290c2047759b6fe74884903dbf9c73a11cc77",
+      "lastUpdated": "2023-09-03T04:48:41",
+      "url": "https://huggingface.co/BlinkDL/rwkv-5-music/blob/main/RWKV-5-MIDI-560M-v1-20230902-ctx4096.pth",
+      "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-5-music/resolve/main/RWKV-5-MIDI-560M-v1-20230902-ctx4096.pth"
    }
  ]
 }
Author	SHA1	Message	Date
josc146	97ae139de5	release v1.4.9	2023-10-27 14:03:28 +08:00
josc146	afd15ef2c5	base64 preset support	2023-10-27 13:35:29 +08:00
josc146	6c73eae9f6	edited chat message now is marked as Normal	2023-10-27 13:11:12 +08:00
josc146	7078f47f72	allow avatarImg to be local absolute path	2023-10-27 12:53:20 +08:00
josc146	d43954cc88	improve message interruption and retry for Chat page	2023-10-27 12:13:05 +08:00
josc146	c87de93498	allow conversation with some document (.pdf, .txt)	2023-10-27 11:36:29 +08:00
josc146	810843a5ab	update manifest.json	2023-10-27 00:48:37 +08:00
josc146	f7cbd2c803	update manifest.json	2023-10-26 18:04:06 +08:00
josc146	faf1852012	update stop strategy	2023-10-26 17:47:40 +08:00
josc146	43cfab5d4b	change default World series prefix to User/Assistant	2023-10-26 16:58:53 +08:00
josc146	627a20936d	RWKVType now no longer relies on the file name	2023-10-26 16:55:33 +08:00
josc146	1d7f19ffaf	update sample.jsonl	2023-10-26 14:08:16 +08:00
josc146	d80565d780	mark rwkv raven series as old model	2023-10-26 13:32:59 +08:00
josc146	d7ba88953d	chore	2023-10-25 22:53:14 +08:00
josc146	30e1c3171e	update kernel (CUDA Compute Capability 5.3)	2023-10-25 22:53:14 +08:00
josc146	1f058b16ac	update kernel (CUDA Compute Capability 6.1, Previously 7.5)	2023-10-25 22:53:13 +08:00
josc146	4a192f4057	upgrade to webgpu 0.2.2 (https://github.com/josStorer/ai00_rwkv_server )	2023-10-25 21:02:44 +08:00
josc146	0331bf47f7	upgrade rwkv 0.8.16 (DirectML support; rwkv 5.2 no longer needs to ensure custom cuda kernel enabled)	2023-10-25 17:56:18 +08:00
josc146	2acdaa96b2	chore	2023-10-25 17:51:59 +08:00
josc146	1d200d53ab	fix beta linux kernel	2023-10-25 17:51:13 +08:00
josc146	df9e1f408e	add `/file-to-text` api	2023-10-25 17:14:33 +08:00
josc146	4a18696686	add pip --no-warn-script-location	2023-10-25 17:08:50 +08:00
josc146	46b3b285f5	upgrade packages	2023-10-25 17:07:40 +08:00
josc146	1d6aeab9dc	fix the `make` command on Linux and macOS, no longer need manual operations on the wsl.go file. (#158 , #173 , #207 )	2023-10-25 16:12:34 +08:00
josc146	ab110ba30b	chore	2023-10-24 23:41:18 +08:00
josc146	2f0fa4ee56	update readme	2023-10-24 21:11:55 +08:00
josc146	0005816c1d	fix linux kernel (partial revert `68228a45`)	2023-10-05 00:08:18 +08:00
josc146	f70672e5a0	update .gitignore	2023-10-05 00:08:02 +08:00
github-actions[bot]	ee057071a5	release v1.4.8	2023-10-03 07:05:41 +00:00