Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

limit number of available GPUs #61

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

junshi15
Copy link

@junshi15 junshi15 commented Apr 19, 2016

This PR limits the number of GPUs available to BIDMat.

It solves the following issue. In a shared environment (grid, cloud, etc), the grid admin may grant a user a number of GPUs for the BIDMat job, however the physical IDs of the available GPUs are not given. The user need to find them first, then limit BIDMat to use those GPUs.

For example, say there are 8 GPUs (No. 0~No. 7) in a node, among them No. 0, No. 1, No. 3 are already used and not available to new jobs. 2 GPUs are granted to a new BIDMat job, but the program has to search for the available GPUs and limit itself to those. In this case, No. 2 and No. 4 are a pair available for this job.

Setting CUDA_VISIBLE_DEVICES is not an options since the GPUs are not known a priori.

In this PR, we store (physical device ID <--> logical device ID) map and convert between the two indices during setGPU and getGPU. In the example above:
physical device No. 2 <--> logical device No. 0
physical device No. 4 <--> logical device No. 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant