From 3a00bbf44d336fcfafba100ba780fe6623a81c90 Mon Sep 17 00:00:00 2001 From: josc146 Date: Sat, 9 Dec 2023 12:56:15 +0800 Subject: [PATCH] update readme --- README.md | 113 +++++++++++++++++++++++++++++++++++++++++++-------- README_JA.md | 95 +++++++++++++++++++++++++++++++++++++++---- README_ZH.md | 4 +- 3 files changed, 184 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index 137cc4b..dd768be 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ English | [简体中文](README_ZH.md) | [日本語](README_JA.md) [![MacOS][MacOS-image]][MacOS-url] [![Linux][Linux-image]][Linux-url] -[FAQs](https://github.com/josStorer/RWKV-Runner/wiki/FAQs) | [Preview](#Preview) | [Download][download-url] | [Server-Deploy-Examples](https://github.com/josStorer/RWKV-Runner/tree/master/deploy-examples) +[FAQs](https://github.com/josStorer/RWKV-Runner/wiki/FAQs) | [Preview](#Preview) | [Download][download-url] | [Simple Deploy Example](#Simple-Deploy-Example) | [Server Deploy Examples](https://github.com/josStorer/RWKV-Runner/tree/master/deploy-examples) | [MIDI Hardware Input](#MIDI-Input) [license-image]: http://img.shields.io/badge/license-MIT-blue.svg @@ -57,20 +57,49 @@ English | [简体中文](README_ZH.md) | [日本語](README_JA.md) ## Features -- RWKV model management and one-click startup -- Fully compatible with the OpenAI API, making every ChatGPT client an RWKV client. After starting the model, +- RWKV model management and one-click startup. +- Front-end and back-end separation, if you don't want to use the client, also allows for separately deploying the + front-end service, or the back-end inference service, or the back-end inference service with a WebUI. + [Simple Deploy Example](#Simple-Deploy-Example) | [Server Deploy Examples](https://github.com/josStorer/RWKV-Runner/tree/master/deploy-examples) +- Compatible with the OpenAI API, making every ChatGPT client an RWKV client. After starting the model, open http://127.0.0.1:8000/docs to view more details. -- Automatic dependency installation, requiring only a lightweight executable program -- Configs with 2G to 32G VRAM are included, works well on almost all computers -- User-friendly chat and completion interaction interface included -- Easy-to-understand and operate parameter configuration -- Built-in model conversion tool -- Built-in download management and remote model inspection -- Built-in one-click LoRA Finetune -- Can also be used as an OpenAI ChatGPT and GPT-Playground client -- Multilingual localization -- Theme switching -- Automatic updates +- Automatic dependency installation, requiring only a lightweight executable program. +- Pre-set multi-level VRAM configs, works well on almost all computers. In Configs page, switch Strategy to WebGPU, it + can also run on AMD, Intel, and other graphics cards. +- User-friendly chat, completion, and composition interaction interface included. Also supports chat presets, attachment + uploads, MIDI hardware input, and track editing. + [Preview](#Preview) | [MIDI Hardware Input](#MIDI-Input) +- Built-in WebUI option, one-click start of Web service, sharing your hardware resources. +- Easy-to-understand and operate parameter configuration, along with various operation guidance prompts. +- Built-in model conversion tool. +- Built-in download management and remote model inspection. +- Built-in one-click LoRA Finetune. +- Can also be used as an OpenAI ChatGPT and GPT-Playground client. (Fill in the API URL and API Key in Settings page) +- Multilingual localization. +- Theme switching. +- Automatic updates. + +## Simple Deploy Example + +```bash +git clone https://github.com/josStorer/RWKV-Runner + +# Then +cd RWKV-Runner +python ./backend-python/main.py #The backend inference service has been started, request /switch-model API to load the model, refer to the API documentation: http://127.0.0.1:8000/docs + +# Or +cd RWKV-Runner/frontend +npm ci +npm run build #Compile the frontend +cd .. +python ./backend-python/webui_server.py #Start the frontend service separately +# Or +python ./backend-python/main.py --webui #Start the frontend and backend service at the same time + +# Help Info +python ./backend-python/main.py -h +``` ## API Concurrency Stress Testing @@ -133,6 +162,48 @@ for i in np.argsort(embeddings_cos_sim)[::-1]: print(f"{embeddings_cos_sim[i]:.10f} - {values[i]}") ``` +## MIDI Input + +Tip: You can download https://github.com/josStorer/sgm_plus and unzip it to the program's `assets/sound-font` directory +to use it as an offline sound source. Please note that if you are compiling the program from source code, do not place +it in the source code directory. + +### USB MIDI Connection + +- USB MIDI devices are plug-and-play, and you can select your input device in the Composition page +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/13bb92c3-4504-482d-ab82-026ac6c31095) + +### Mac MIDI Bluetooth Connection + +- For Mac users who want to use Bluetooth input, + please install [Bluetooth MIDI Connect](https://apps.apple.com/us/app/bluetooth-midi-connect/id1108321791), then click + the tray icon to connect after launching, + afterwards, you can select your input device in the Composition page. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/c079a109-1e3d-45c1-bbf5-eed85da1550e) + +### Windows MIDI Bluetooth Connection + +- Windows seems to have implemented Bluetooth MIDI support only for UWP (Universal Windows Platform) apps. Therefore, it + requires multiple steps to establish a connection. We need to create a local virtual MIDI device and then launch a UWP + application. Through this UWP application, we will redirect Bluetooth MIDI input to the virtual MIDI device, and then + this software will listen to the input from the virtual MIDI device. +- So, first, you need to + download [loopMIDI](https://www.tobias-erichsen.de/wp-content/uploads/2020/01/loopMIDISetup_1_0_16_27.zip) + to create a virtual MIDI device. Click the plus sign in the bottom left corner to create the device. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/b75998ff-115c-4ddd-b97c-deeb5c106255) +- Next, you need to download [Bluetooth LE Explorer](https://apps.microsoft.com/detail/9N0ZTKF1QD98) to discover and + connect to Bluetooth MIDI devices. Click "Start" to search for devices, and then click "Pair" to bind the MIDI device. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/c142c3ea-a973-4531-9807-4c385d640a2b) +- Finally, you need to install [MIDIberry](https://apps.microsoft.com/detail/9N39720H2M05), + This UWP application can redirect Bluetooth MIDI input to the virtual MIDI device. After launching it, double-click + your actual Bluetooth MIDI device name in the input field, and in the output field, double-click the virtual MIDI + device name we created earlier. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/5ad6a1d9-4f68-4d95-ae17-4296107d1669) +- Now, you can select the virtual MIDI device as the input in the Composition page. Bluetooth LE Explorer no longer + needs to run, and you can also close the loopMIDI window, it will run automatically in the background. Just keep + MIDIberry open. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/1c371821-c7b7-4c18-8e42-9e315efbe427) + ## Related Repositories: - RWKV-4-World: https://huggingface.co/BlinkDL/rwkv-4-world/tree/main @@ -146,27 +217,35 @@ for i in np.argsort(embeddings_cos_sim)[::-1]: ### Homepage -![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/d7f24d80-f382-428d-8b28-edf87e1549e2) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/c9b9cdd0-63f9-4319-9f74-5bf5d7df5a67) ### Chat ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/80009872-528f-4932-aeb2-f724fa892e7c) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/e98c9038-3323-47b0-8edb-d639fafd37b2) + ### Completion ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/bf49de8e-3b89-4543-b1ef-7cd4b19a1836) ### Composition +Tip: You can download https://github.com/josStorer/sgm_plus and unzip it to the program's `assets/sound-font` directory +to use it as an offline sound source. Please note that if you are compiling the program from source code, do not place +it in the source code directory. + ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/e8ad908d-3fd2-4e92-bcdb-96815cb836ee) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/b2ce4761-9e75-477e-a182-d0255fb8ac76) + ### Configuration -![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/48befdc6-e03c-4851-9bee-22f77ee2640e) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/f41060dc-5517-44af-bb3f-8ef71720016d) ### Model Management -![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/367fe4f8-cc12-475f-9371-3cf62cdbf293) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/b1581147-a6ce-4493-8010-e33c0ddeca0a) ### Download Management diff --git a/README_JA.md b/README_JA.md index 2bdca23..70c7ef2 100644 --- a/README_JA.md +++ b/README_JA.md @@ -21,7 +21,7 @@ [![MacOS][MacOS-image]][MacOS-url] [![Linux][Linux-image]][Linux-url] -[FAQs](https://github.com/josStorer/RWKV-Runner/wiki/FAQs) | [プレビュー](#Preview) | [ダウンロード][download-url] | [サーバーデプロイ例](https://github.com/josStorer/RWKV-Runner/tree/master/deploy-examples) +[FAQs](https://github.com/josStorer/RWKV-Runner/wiki/FAQs) | [プレビュー](#Preview) | [ダウンロード][download-url] | [シンプルなデプロイの例](#Simple-Deploy-Example) | [サーバーデプロイ例](https://github.com/josStorer/RWKV-Runner/tree/master/deploy-examples) | [MIDIハードウェア入力](#MIDI-Input) [license-image]: http://img.shields.io/badge/license-MIT-blue.svg @@ -58,20 +58,47 @@ ## 特徴 - RWKV モデル管理とワンクリック起動 -- OpenAI API と完全に互換性があり、すべての ChatGPT クライアントを RWKV クライアントにします。モデル起動後、 +- フロントエンドとバックエンドの分離は、クライアントを使用しない場合でも、フロントエンドサービス、またはバックエンド推論サービス、またはWebUIを備えたバックエンド推論サービスを個別に展開することを可能にします。 + [シンプルなデプロイの例](#Simple-Deploy-Example) | [サーバーデプロイ例](https://github.com/josStorer/RWKV-Runner/tree/master/deploy-examples) +- OpenAI API と互換性があり、すべての ChatGPT クライアントを RWKV クライアントにします。モデル起動後、 http://127.0.0.1:8000/docs を開いて詳細をご覧ください。 - 依存関係の自動インストールにより、軽量な実行プログラムのみを必要とします -- 2G から 32G の VRAM のコンフィグが含まれており、ほとんどのコンピュータで動作します -- ユーザーフレンドリーなチャットと完成インタラクションインターフェースを搭載 -- 分かりやすく操作しやすいパラメータ設定 +- 事前設定された多段階のVRAM設定、ほとんどのコンピュータで動作します。配置ページで、ストラテジーをWebGPUに切り替えると、AMD、インテル、その他のグラフィックカードでも動作します +- ユーザーフレンドリーなチャット、完成、および作曲インターフェイスが含まれています。また、チャットプリセット、添付ファイルのアップロード、MIDIハードウェア入力、トラック編集もサポートしています。 + [プレビュー](#Preview) | [MIDIハードウェア入力](#MIDI-Input) +- 内蔵WebUIオプション、Webサービスのワンクリック開始、ハードウェアリソースの共有 +- 分かりやすく操作しやすいパラメータ設定、各種操作ガイダンスプロンプトとともに - 内蔵モデル変換ツール - ダウンロード管理とリモートモデル検査機能内蔵 - 内蔵のLoRA微調整機能を搭載しています -- このプログラムは、OpenAI ChatGPTとGPT Playgroundのクライアントとしても使用できます +- このプログラムは、OpenAI ChatGPTとGPT Playgroundのクライアントとしても使用できます（設定ページで `API URL` と `API Key` + を入力してください） - 多言語ローカライズ - テーマ切り替え - 自動アップデート +## Simple Deploy Example + +```bash +git clone https://github.com/josStorer/RWKV-Runner + +# Then +cd RWKV-Runner +python ./backend-python/main.py #The backend inference service has been started, request /switch-model API to load the model, refer to the API documentation: http://127.0.0.1:8000/docs + +# Or +cd RWKV-Runner/frontend +npm ci +npm run build #Compile the frontend +cd .. +python ./backend-python/webui_server.py #Start the frontend service separately +# Or +python ./backend-python/main.py --webui #Start the frontend and backend service at the same time + +# Help Info +python ./backend-python/main.py -h +``` + ## API 同時実行ストレステスト ```bash @@ -134,6 +161,48 @@ for i in np.argsort(embeddings_cos_sim)[::-1]: print(f"{embeddings_cos_sim[i]:.10f} - {values[i]}") ``` +## MIDI Input + +Tip: You can download https://github.com/josStorer/sgm_plus and unzip it to the program's `assets/sound-font` directory +to use it as an offline sound source. Please note that if you are compiling the program from source code, do not place +it in the source code directory. + +### USB MIDI Connection + +- USB MIDI devices are plug-and-play, and you can select your input device in the Composition page +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/13bb92c3-4504-482d-ab82-026ac6c31095) + +### Mac MIDI Bluetooth Connection + +- For Mac users who want to use Bluetooth input, + please install [Bluetooth MIDI Connect](https://apps.apple.com/us/app/bluetooth-midi-connect/id1108321791), then click + the tray icon to connect after launching, + afterwards, you can select your input device in the Composition page. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/c079a109-1e3d-45c1-bbf5-eed85da1550e) + +### Windows MIDI Bluetooth Connection + +- Windows seems to have implemented Bluetooth MIDI support only for UWP (Universal Windows Platform) apps. Therefore, it + requires multiple steps to establish a connection. We need to create a local virtual MIDI device and then launch a UWP + application. Through this UWP application, we will redirect Bluetooth MIDI input to the virtual MIDI device, and then + this software will listen to the input from the virtual MIDI device. +- So, first, you need to + download [loopMIDI](https://www.tobias-erichsen.de/wp-content/uploads/2020/01/loopMIDISetup_1_0_16_27.zip) + to create a virtual MIDI device. Click the plus sign in the bottom left corner to create the device. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/b75998ff-115c-4ddd-b97c-deeb5c106255) +- Next, you need to download [Bluetooth LE Explorer](https://apps.microsoft.com/detail/9N0ZTKF1QD98) to discover and + connect to Bluetooth MIDI devices. Click "Start" to search for devices, and then click "Pair" to bind the MIDI device. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/c142c3ea-a973-4531-9807-4c385d640a2b) +- Finally, you need to install [MIDIberry](https://apps.microsoft.com/detail/9N39720H2M05), + This UWP application can redirect Bluetooth MIDI input to the virtual MIDI device. After launching it, double-click + your actual Bluetooth MIDI device name in the input field, and in the output field, double-click the virtual MIDI + device name we created earlier. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/5ad6a1d9-4f68-4d95-ae17-4296107d1669) +- Now, you can select the virtual MIDI device as the input in the Composition page. Bluetooth LE Explorer no longer + needs to run, and you can also close the loopMIDI window, it will run automatically in the background. Just keep + MIDIberry open. +- ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/1c371821-c7b7-4c18-8e42-9e315efbe427) + ## 関連リポジトリ: - RWKV-4-World: https://huggingface.co/BlinkDL/rwkv-4-world/tree/main @@ -147,27 +216,35 @@ for i in np.argsort(embeddings_cos_sim)[::-1]: ### ホームページ -![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/d7f24d80-f382-428d-8b28-edf87e1549e2) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/c9b9cdd0-63f9-4319-9f74-5bf5d7df5a67) ### チャット ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/80009872-528f-4932-aeb2-f724fa892e7c) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/e98c9038-3323-47b0-8edb-d639fafd37b2) + ### 補完 ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/bf49de8e-3b89-4543-b1ef-7cd4b19a1836) ### 作曲 +Tip: You can download https://github.com/josStorer/sgm_plus and unzip it to the program's `assets/sound-font` directory +to use it as an offline sound source. Please note that if you are compiling the program from source code, do not place +it in the source code directory. + ![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/e8ad908d-3fd2-4e92-bcdb-96815cb836ee) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/b2ce4761-9e75-477e-a182-d0255fb8ac76) + ### コンフィグ -![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/48befdc6-e03c-4851-9bee-22f77ee2640e) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/f41060dc-5517-44af-bb3f-8ef71720016d) ### モデル管理 -![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/367fe4f8-cc12-475f-9371-3cf62cdbf293) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/b1581147-a6ce-4493-8010-e33c0ddeca0a) ### ダウンロード管理 diff --git a/README_ZH.md b/README_ZH.md index 349bfc4..46102ac 100644 --- a/README_ZH.md +++ b/README_ZH.md @@ -61,7 +61,7 @@ API兼容的接口，这意味着一切ChatGPT客户端都是RWKV客户端。 [简明服务部署示例](#Simple-Deploy-Example) | [服务器部署示例](https://github.com/josStorer/RWKV-Runner/tree/master/deploy-examples) - 与OpenAI API兼容，一切ChatGPT客户端，都是RWKV客户端。启动模型后，打开 http://127.0.0.1:8000/docs 查看API文档 - 全自动依赖安装，你只需要一个轻巧的可执行程序 -- 预设多级显存配置，几乎在各种电脑上工作良好。通过配置页面切换到WebGPU策略，还可以在AMD，Intel等显卡上运行 +- 预设多级显存配置，几乎在各种电脑上工作良好。通过配置页面切换Strategy到WebGPU，还可以在AMD，Intel等显卡上运行 - 自带用户友好的聊天，续写，作曲交互页面。支持聊天预设，附件上传，MIDI硬件输入及音轨编辑。 [预览](#Preview) | [MIDI硬件输入](#MIDI-Input) - 内置WebUI选项，一键启动Web服务，共享硬件资源 @@ -230,7 +230,7 @@ for i in np.argsort(embeddings_cos_sim)[::-1]: ### 模型管理 -![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/6bbbecec-d000-457e-993d-27ff53881d09) +![image](https://github.com/josStorer/RWKV-Runner/assets/13366013/871f2d2a-7e41-4be7-9b32-be1b3e00dc3e) ### 下载管理