Lemonade

🍋 Lemonade SDK: Quickly serve, benchmark and deploy LLMs

The Lemonade SDK is designed to make it easy to serve, benchmark, and deploy large language models (LLMs) on a variety of hardware platforms, including CPU, GPU, and NPU.

The Lemonade SDK is comprised of the following:

• 🌐 Lemonade Server: A server interface that uses the standard Open AI API, allowing applications to integrate with local LLMs.
• 🐍 Lemonade Python API: Offers High-Level API for easy integration of Lemonade LLMs into Python applications and Low-Level API for custom experiments.
• 🖥️ Lemonade CLI: The lemonade CLI lets you mix-and-match LLMs, frameworks (PyTorch, ONNX, GGUF), and measurement tools to run experiments. The available tools are:
  • Prompting an LLM.
  • Measuring the accuracy of an LLM using a variety of tests.
  • Benchmarking an LLM to get the time-to-first-token and tokens per second.
  • Profiling the memory usage of an LLM.

Click here to get started with Lemonade.

Contributing

We are actively seeking collaborators from across the industry. If you would like to contribute to this project, please check out our contribution guide.

Maintainers

This project is sponsored by AMD. It is maintained by @danielholanda @jeremyfowers @ramkrishna @vgodsoe in equal measure. You can reach us by filing an issue or email lemonade@amd.com.

License

This project is licensed under the Apache 2.0 License. Portions of the project are licensed as described in NOTICE.md.