theluyuan/RWKV-Runner
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

add pre-compiled beta cuda kernel (rwkv-beta==0.8.5, 40%+ faster for fp16) (thanks to #180, pre-compiled kernel of RTX 40 Series will be included later)

Browse Source
This commit is contained in:
josc146 2023-09-18 23:02:49 +08:00
parent 5e5e1e9651
commit d7abe5f0d1
3 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/release.yml vendored
View File

@ -108,6 +108,7 @@ jobs:
cd .. cd ..
go install github.com/wailsapp/wails/v2/cmd/wails@latest go install github.com/wailsapp/wails/v2/cmd/wails@latest
rm -rf ./backend-python/wkv_cuda_utils rm -rf ./backend-python/wkv_cuda_utils
rm ./backend-python/rwkv_pip/beta/wkv_cuda.pyd
rm ./backend-python/get-pip.py rm ./backend-python/get-pip.py
sed -i '1,2d' ./backend-golang/wsl_not_windows.go sed -i '1,2d' ./backend-golang/wsl_not_windows.go
rm ./backend-golang/wsl.go rm ./backend-golang/wsl.go
@ -140,6 +141,7 @@ jobs:
cd .. cd ..
go install github.com/wailsapp/wails/v2/cmd/wails@latest go install github.com/wailsapp/wails/v2/cmd/wails@latest
rm -rf ./backend-python/wkv_cuda_utils rm -rf ./backend-python/wkv_cuda_utils
rm ./backend-python/rwkv_pip/beta/wkv_cuda.pyd
rm ./backend-python/get-pip.py rm ./backend-python/get-pip.py
sed -i '' '1,2d' ./backend-golang/wsl_not_windows.go sed -i '' '1,2d' ./backend-golang/wsl_not_windows.go
rm ./backend-golang/wsl.go rm ./backend-golang/wsl.go

1
backend-python/rwkv_pip/beta/model.py vendored
View File

@ -94,6 +94,7 @@ if os.environ.get("RWKV_CUDA_ON") == "1":
f"{current_path}/cuda/att_one_v5.cu", f"{current_path}/cuda/att_one_v5.cu",
], ],
verbose=True, verbose=True,
extra_ldflags=["cublas.lib"],
extra_cuda_cflags=[ extra_cuda_cflags=[
"-t 4", "-t 4",
"-std=c++17", "-std=c++17",

BIN
backend-python/rwkv_pip/beta/wkv_cuda.pyd vendored Normal file
View File

Binary file not shown.