Skip to content

Commit

Permalink
Build from source instructions (#38)
Browse files Browse the repository at this point in the history
* Build from source instructions

* updated build from source instructions
  • Loading branch information
laikhtewari authored Dec 15, 2023
1 parent fc57497 commit c141abd
Showing 1 changed file with 18 additions and 3 deletions.
21 changes: 18 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,32 @@ You can use a Docker container to try Optimum-NVIDIA today. Images are available
docker pull huggingface/optimum-nvidia
```

<!-- You can also build from source with the Dockerfile provided here.
An Optimum-NVIDIA package that can be installed with `pip` will be made available soon.

## Building from source
<!---
Currently, TRT LLM is built and run with Docker, so we should wait until pip installation is available;
Ideally the user doesn't need to use docker at all to build from source, they should be able to run something like
`git clone [...] && pip install -e optimum-nvidia`
-->

Instead of using the pre-built docker container, you can build Optimum-NVIDIA from source:
```bash
TARGET_SM = "90-real,89-real"
git clone --recursive --depth=1 [email protected]:huggingface/optimum-nvidia.git
cd optimum-nvidia/third-party/tensorrt-llm
make -C docker release_build CUDA_ARCHS=$TARGET_SM
cd ../.. && docker build -t <organisation_name/image_name>:<version> -f docker/Dockerfile .
```

<!--
```bash
git clone [email protected]:huggingface/optimum-nvidia.git
cd optimum-nvidia
docker build Dockerfile
docker run optimum-nvidia
``` -->

An Optimum-NVIDIA package that can be installed with `pip` will be made available soon.

# Quickstart Guide
## Pipelines

Expand Down

0 comments on commit c141abd

Please sign in to comment.