{ "version": "1.0.0", "introduction": { "en": "RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the \"GPT\" mode to quickly compute the hidden state for the \"RNN\" mode.
So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, \"infinite\" ctx_len, and free sentence embedding (using the final hidden state).", "zh": "RWKV是具有Transformer级别LLM性能的RNN,也可以像GPT Transformer一样直接进行训练(可并行化)。而且它是100% attention-free的。你只需在位置t处获得隐藏状态即可计算位置t + 1处的状态。你可以使用“GPT”模式快速计算用于“RNN”模式的隐藏状态。
因此,它将RNN和Transformer的优点结合起来 - 高性能、快速推理、节省显存、快速训练、“无限”上下文长度以及免费的语句嵌入(使用最终隐藏状态)。" }, "about": { "en": "
\n\nProject Source Code:\nhttps://github.com/josStorer/RWKV-Runner\nAuthor: [@josStorer](https://github.com/josStorer)\n\nRelated Repositories:\nRWKV-4-Raven: https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main\nChatRWKV: https://github.com/BlinkDL/ChatRWKV\nRWKV-LM: https://github.com/BlinkDL/RWKV-LM\n\n
", "zh": "
\n\n本项目源码:\nhttps://github.com/josStorer/RWKV-Runner\n作者: [@josStorer](https://github.com/josStorer)\n\n相关仓库:\nRWKV-4-Raven: https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main\nChatRWKV: https://github.com/BlinkDL/ChatRWKV\nRWKV-LM: https://github.com/BlinkDL/RWKV-LM\n\n
" }, "localModelDir": "models", "programFiles": [ { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/requirements.txt", "path": "backend-python/requirements.txt" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/requirements_versions.txt", "path": "backend-python/requirements_versions.txt" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/main.py", "path": "backend-python/main.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/global_var.py", "path": "backend-python/global_var.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/convert_model.py", "path": "backend-python/convert_model.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/dep_check.py", "path": "backend-python/dep_check.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/routes/completion.py", "path": "backend-python/routes/completion.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/routes/config.py", "path": "backend-python/routes/config.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/utils/ngrok.py", "path": "backend-python/utils/ngrok.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/utils/rwkv.py", "path": "backend-python/utils/rwkv.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/utils/torch.py", "path": "backend-python/utils/torch.py" }, { "url": "https://cdn.jsdelivr.net/gh/josstorer/RWKV-Runner/backend-python/20B_tokenizer.json", "path": "backend-python/20B_tokenizer.json" }, { "url": "https://bootstrap.pypa.io/get-pip.py", "path": "backend-python/get-pip.py" } ], "models": [ { "name": "RWKV-4-Raven-3B-v11-Eng99%-Other1%-20230425-ctx4096.pth", "desc": { "en": "English 3B v11", "zh": "英文 3B v11" }, "size": 5969345074, "SHA256": "982ad3d794efe58992db23c6d694c57a9e62d54718264ec6d6acfae5eb0eea12", "lastUpdated": "2023-04-26T14:27:55", "url": "https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-3B-v11-Eng99%25-Other1%25-20230425-ctx4096.pth", "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-raven/resolve/main/RWKV-4-Raven-3B-v11-Eng99%25-Other1%25-20230425-ctx4096.pth" }, { "name": "RWKV-4-Raven-7B-v10x-Eng49%-Chn50%-Other1%-20230423-ctx4096.pth", "desc": { "en": "Chinese 7B v10x", "zh": "中文 7B v10x" }, "size": 14785389874, "SHA256": "7aaf40bb3d440a949db3a146b0a5bbb3e925942b496775b51f5630a582fc236d", "lastUpdated": "2023-04-24T07:48:55", "url": "https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-7B-v10x-Eng49%25-Chn50%25-Other1%25-20230423-ctx4096.pth", "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-raven/resolve/main/RWKV-4-Raven-7B-v10x-Eng49%25-Chn50%25-Other1%25-20230423-ctx4096.pth" }, { "name": "RWKV-4-Raven-7B-v11-Eng49%-Chn49%-Jpn1%-Other1%-20230430-ctx8192.pth", "desc": { "en": "Chinese 7B v11", "zh": "中文 7B v11" }, "size": 14785389874, "SHA256": "9e67a74964abcb4463711e447ddf47735561d7b40592d2d02b29d2e796a4fd14", "lastUpdated": "2023-04-30T14:35:59", "url": "https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-7B-v11-Eng49%25-Chn49%25-Jpn1%25-Other1%25-20230430-ctx8192.pth", "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-raven/resolve/main/RWKV-4-Raven-7B-v11-Eng49%25-Chn49%25-Jpn1%25-Other1%25-20230430-ctx8192.pth" }, { "name": "RWKV-4-Raven-7B-v11x-Eng99%-Other1%-20230429-ctx8192.pth", "desc": { "en": "English 7B v11x", "zh": "英文 7B v11x" }, "size": 14785389874, "SHA256": "f00d5c75b453f2b20ad875fb5a324564c34024eea25a015f5eb441e4f364c3fe", "lastUpdated": "2023-04-29T11:44:32", "url": "https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-7B-v11x-Eng99%25-Other1%25-20230429-ctx8192.pth", "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-raven/resolve/main/RWKV-4-Raven-7B-v11x-Eng99%25-Other1%25-20230429-ctx8192.pth" }, { "name": "RWKV-4-Raven-1B5-v11-Eng99%-Other1%-20230425-ctx4096.pth", "desc": { "en": "English 1.5B v11", "zh": "英文 1.5B v11" }, "size": 3030279730, "SHA256": "4ac715aecc5b1c90e8e37eebb8163392699066ec23b18144416e91cb4e78675a", "lastUpdated": "2023-04-26T14:27:55", "url": "https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-1B5-v11-Eng99%25-Other1%25-20230425-ctx4096.pth", "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-raven/resolve/main/RWKV-4-Raven-1B5-v11-Eng99%25-Other1%25-20230425-ctx4096.pth" }, { "name": "RWKV-4-Raven-3B-v11-Eng49%-Chn49%-Jpn1%-Other1%-20230429-ctx4096.pth", "desc": { "en": "Chinese 3B v11", "zh": "中文 3B v11" }, "size": 5969345074, "SHA256": "af12300d9875e0e166c23d6e9b20928db435073060bf1d36f874060de92ada98", "lastUpdated": "2023-04-29T11:51:51", "url": "https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-3B-v11-Eng49%25-Chn49%25-Jpn1%25-Other1%25-20230429-ctx4096.pth", "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-raven/resolve/main/RWKV-4-Raven-3B-v11-Eng49%25-Chn49%25-Jpn1%25-Other1%25-20230429-ctx4096.pth" }, { "name": "RWKV-4-Raven-14B-v11x-Eng99%-Other1%-20230501-ctx8192.pth", "desc": { "en": "English 14B v11x", "zh": "英文 14B v11x" }, "size": 28297309490, "SHA256": "c4bc72406c3c62613e8e2592e8d07ac045f8a88381c728f8eb60af890e299f4d", "lastUpdated": "2023-05-02T09:43:33", "url": "https://huggingface.co/BlinkDL/rwkv-4-raven/blob/main/RWKV-4-Raven-14B-v11x-Eng99%25-Other1%25-20230501-ctx8192.pth", "downloadUrl": "https://huggingface.co/BlinkDL/rwkv-4-raven/resolve/main/RWKV-4-Raven-14B-v11x-Eng99%25-Other1%25-20230501-ctx8192.pth" } ] }