2023-05-21 17:28:51 +08:00
< p align = "center" >
< img src = "https://github.com/josStorer/RWKV-Runner/assets/13366013/d24834b0-265d-45f5-93c0-fac1e19562af" >
< / p >
2023-05-03 23:38:54 +08:00
2023-05-21 17:28:51 +08:00
< h1 align = "center" > RWKV Runner< / h1 >
< div align = "center" >
2023-05-22 11:48:56 +08:00
This project aims to eliminate the barriers of using large language models by automating everything for you. All you
need is a lightweight executable program of just a few megabytes. Additionally, this project provides an interface
compatible with the OpenAI API, which means that every ChatGPT client is an RWKV client.
2023-05-21 17:28:51 +08:00
[![license][license-image]][license-url]
[![release][release-image]][release-url]
English | [简体中文 ](README_ZH.md )
[Preview ](#Preview ) | [Download][download-url]
[license-image]: http://img.shields.io/badge/license-MIT-blue.svg
[license-url]: https://github.com/josStorer/RWKV-Runner/blob/master/LICENSE
[release-image]: https://img.shields.io/github/release/josStorer/RWKV-Runner.svg
[release-url]: https://github.com/josStorer/RWKV-Runner/releases/latest
2023-05-23 16:45:53 +08:00
[download-url]: https://github.com/josStorer/RWKV-Runner/releases/download/v1.0.2/RWKV-Runner_windows_x64.exe
2023-05-21 17:28:51 +08:00
< / div >
2023-05-23 20:52:18 +08:00
#### Default configs do not enable custom CUDA kernel acceleration, but I strongly recommend that you enable it and run with int8 precision, which is much faster and consumes much less VRAM. Go to the Configs page and turn on `Use Custom CUDA kernel to Accelerate`.
2023-05-23 21:25:33 +08:00
#### For different tasks, adjusting API parameters can achieve better results. For example, for translation tasks, you can try setting Temperature to 1 and Top_P to 0.3.
2023-05-21 17:28:51 +08:00
## Features
- RWKV model management and one-click startup
2023-05-22 11:48:56 +08:00
- Fully compatible with the OpenAI API, making every ChatGPT client an RWKV client. After starting the model,
open http://127.0.0.1:8000/docs to view more details.
2023-05-21 17:28:51 +08:00
- Automatic dependency installation, requiring only a lightweight executable program
- User-friendly chat interaction interface included
- Easy-to-understand and operate parameter configuration
- Built-in model conversion tool
- Built-in download management and remote model inspection
- Multilingual localization
- Theme switching
- Automatic updates
## Todo
2023-05-23 14:58:45 +08:00
- [ ] Model training functionality
- [x] CUDA operator int8 acceleration
- [ ] macOS support
- [ ] Linux support
2023-05-21 17:28:51 +08:00
## Related Repositories:
- RWKV-4-Raven: https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main
- ChatRWKV: https://github.com/BlinkDL/ChatRWKV
- RWKV-LM: https://github.com/BlinkDL/RWKV-LM
## Preview
### Homepage
2023-05-21 17:43:23 +08:00
2023-05-21 17:28:51 +08:00
### Chat
2023-05-21 17:43:23 +08:00
2023-05-21 17:28:51 +08:00
2023-05-24 21:45:50 +08:00
### Completion
2023-05-21 17:28:51 +08:00
### Configuration
2023-05-21 17:43:23 +08:00
2023-05-21 17:28:51 +08:00
### Model Management
2023-05-21 17:43:23 +08:00
2023-05-21 17:28:51 +08:00
### Download Management
2023-05-21 17:43:23 +08:00
2023-05-21 17:28:51 +08:00
### Settings
2023-05-21 17:43:23 +08:00
