Running Jobs and Using the Scheduler

You are not logged in.
Please Log In for full access to the web site.
Note that this link will take you to an external site (https://petrock.mit.edu) to authenticate, and then you will be redirected back to this page.

Review your knowledge of concepts from the Running Jobs and Using the Scheduler module

1) Types of Jobs

Interactive jobs are recommended for:

Testing new code

Running the same script over and over

Long running code

Debugging errors

Interactive data analysis

2) Using the Scheduler

What is the command to see available resources on Engaging?

What is the base command to start an interactive job on the mit_normal partition with default resources?

If your submission script for your serial job is "myscript.sh", what command would you run to submit this as a batch job? For this exercise do not include any command line flags or options.

What command will list all my currently running or pending jobs?

I have a job running with Job ID 12673378. What command would I run to stop this job?br/>

3) Determining Resource Requirements

You have a workflow that processes a large number of files, spread over many sub-directories. Each file is processed independently but requires 2x more than the default RAM per core on the hardware. Submitting this as a Job Array, do you need to add any additional flags?

No, it is good to go.

Yes, I need to ask for additional memory.

Yes, I need to use shared memory, so I may need a full node.

Yes, I need an exclusive node.

You are using a Python package to train a machine learning model. The packages used in your machine learning training use threads but are not distributed, do you need additional flags when submitting the job?

No, it is good to go.

Yes, I need to ask for 2 cores/task to account for the memory requirements.

Yes, I need to use shared memory, so I should request more cores on the same node.

Yes, I need an exclusive node.

4) Batch Job

If you have a throughput application, what is the best tool to use to submit this job in parallel?

Submit many serial jobs separately

An interactive job

A job array

A Jupyter Notebook

How would you submit a parallel distributed memory job? (Select all that apply)

Request tasks using the -n (--ntasks) flag, possibly combined with the -N (--nodes) flag.

Using only the -c flag

Combine the -N (--nodes), and --ntasks-per-node flags

With a Job Array

How would you submit a parallel shared memory job? (Select all that apply)

Combine the -N (--nodes), and --ntasks-per-node flags

Request tasks using the -n (--ntasks) flag, possibly combined with the -N (--nodes) flag.

Using only the -c flag

With a Job Array

True or False: You should always have one Job Array task or process for each input you are running your program on.

True

False

True or false: The flag --gres=gpu:l40s:1 will allocate 1 GPU to each core in your job. If you have 1 core you will get 1 GPU, if you have two cores on the same node you will have 2 GPUs.

True

False

You have profiled your code and found it uses no more than 64GB of RAM. You are running on the Xeon-P8 nodes which have 196 of RAM and 48 cores. You are only running with one process. How many cores should you request?

1 core, it's a single process

8 cores

16 cores

32 cores

True or false: I have not profiled my code and I am using software and/or packages that I did not write. However, I did not write any parallel code in my program so I do not need to request more than a single core.

True

False