> For the complete documentation index, see [llms.txt](https://fall2019.fullstackdeeplearning.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://fall2019.fullstackdeeplearning.com/course-content/infrastructure-and-tooling/resource-management.md). # Resource Management {% embed url="" %} Resource Management - Infrastructure and Tooling {% endembed %} ## Summary * Running complex deep learning models poses a very practical resource management problem: how to give every team the tools they need to train their models without requiring them to operate their own infrastructure? * The most primitive approach is to use **spreadsheets** that allow people to reserve what resources they need to use. * The next approach is to utilize a **SLURM Workload Manager**, a free and open-source job scheduler for Linux and Unix-like kernels. * A very standard approach these days is to use Docker alongside Kubernetes. * **Docker** is a way to package up an entire dependency stack in a lighter-than-a-Virtual-Machine package. * **Kubernetes** is a way to run many Docker containers on top of a cluster. * The last option is to use open-source projects. * Using **Kubeflow** allows you to run model training jobs at scale on containers with the same scalability of container orchestration that comes with Kubernetes. * **Polyaxon** is a self-service multi-user system, taking care of scheduling and managing jobs in order to make the best use of available cluster resources. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://fall2019.fullstackdeeplearning.com/course-content/infrastructure-and-tooling/resource-management.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.