Looking to get an installer and host Lamini on-premise or on a GPU VM in your VPC? Reach out to us at firstname.lastname@example.org!
System requirements 🌾
Before getting started, make sure your machine is set up to run Lamini smoothly. Check that your machine has at least:
- 64 GB CPU memory
- 32 GB GPU memory
- 1 TB disk
- Ubuntu 22*
*other linux distros should work as long as they run Docker/OCI
Reach out to us at email@example.com for advice on configuring and purchasing machines capable of running your desired application, LLM model (e.g. Llama v2), data volume, and number of users.
Our typical configuration is an LLM Superstation, which is a powerful server with multiple GPUs that can fully host, finetune, and deploy the largest models on the Lamini platform. We have configs that fit into an office or datacenter. The Lamini software architecture is based on technologies used in supercomputers and scales horizontally to the largest systems in the world, e.g. more than 10,000 GPUs.
However, you can also run the entire Lamini platform on your laptop. It's a helpful dev/testing environment, and CPUs can run LLMs with hundreds of millions of parameters just fine.
Lamini is entirely self contained and can run on any machine that can run Docker or OCI containers. In addition to the operation system, provisioning involves installing Docker, and installing the GPU driver.
Install Docker by following the instructions here 🔗.
- Install the GPU driver for the operating system following the manufacturer's instructions.
- Run system management interface (SMI) tests inside of a GPU enabled docker container to verify the installation.
Lamini installer 🎁
- Using the link provided by Lamini (reach out if you have any issues!), get the installer:
$ wget -O lamini-installer.sh '
- Add execute permissions:
$ chmod +x lamini-installer.sh.
- Run the installer:
Docs coming soon! Reach out to us at firstname.lastname@example.org!
Start up 🚀
Woo, congrats and welcome to the herd!!
You can also run with
$ ./build-lamini-installer/lamini-installer/lamini-up -d
to start the services as a background daemon process, and view the logs with
Once running, you can checkout the UI at http://localhost:5001!
Most configuration options for the Lamini platform are available in a single
yaml configuration file, which is installed at:
Some common config values:
- verbose : Set to true to enable verbose logging
- powerml : A list of API endpoints. If you want to run different services on different machines, e.g. in a kubernetes cluster, configure each service's endpoints here.
- wandb.key : Add your weight and biases key here to get detailed dashboards for your fine tuning experiments.
- disable_auth : Turn off built in authentication, e.g. if you want to use your own.
- auth.google_whitelist : Set to your company domain to enable any gmail account with a matching domain to log in.
The list of all Lamini services is available in the docker-compose.yaml file
Some common config values:
- USE_HTTPS : enable or disable HTTPS (e.g. for external facing services or internal development)
- volumes.slurm-volume: where do you want fine-tuned models to be stored (they are saved in pytorch format)