Shenzhen Yuanshi Intelligence
Attention-free large language model architecture for efficient AI.
| Date | Investors | Amount | Round |
|---|---|---|---|
* | CNY20.0m | Early VC | |
| Total Funding | 000k | ||
Shenzhen Yuanshi Intelligence Co. is the commercial entity behind RWKV (Receptance Weighted Key Value), an open-source large language model (LLM) architecture. The RWKV project was initiated by developer Bo Peng and later transitioned to a commercialization phase in April 2024 with Peng as CEO and Luo Xuan as COO. The project operates under the Linux Foundation, emphasizing its open-source nature. The company is headquartered in Shenzhen, a city fostering a complete AI ecosystem.
RWKV distinguishes itself by combining the features of Recurrent Neural Networks (RNNs) and Transformers. This hybrid design allows for the parallelizable training found in Transformers while maintaining the efficient inference and constant memory usage characteristic of RNNs. A key feature is its "attention-free" mechanism, which contrasts with standard Transformer models that rely on computationally intensive attention layers. This results in significantly lower computational costs, faster inference, reduced VRAM usage, and the ability to handle theoretically infinite context lengths. The architecture is designed to be chip-friendly and can run on CPUs, making it more accessible than models requiring high-end GPUs.
The company's business model revolves around the commercialization and application of the RWKV architecture. Having secured tens of millions of RMB in an angel funding round in late 2024, Yuanshi Intelligence plans to expand its team, iterate on the architecture, and develop commercial products. The company is exploring applications in diverse sectors such as new energy, consumer devices, and humanoid robotics, aiming to create versatile AI that can adapt to multiple scenarios. The project has fostered a global open-source community with over 400 derivative projects, some of which have achieved funding and profitability.
Keywords: RWKV, large language model, attention-free, Recurrent Neural Network, RNN, Transformer architecture, linear attention, open source AI, efficient inference, Bo Peng, Shenzhen Yuanshi Intelligence, AI architecture, parallelizable training, long context, edge AI, model commercialization, Linux Foundation AI, machine learning, natural language processing, AI for robotics