Every several minutes the global scheduler examines the current backlog of reads and writes. Because the tape library contains a mix of media, the scheduler's first task is to generate a capability matrix that maps pending I/O requests to the read/write compatibilites of drives and tapes. In addition to hardware compatibility, the scheduler also considers some dynamic constraints that can restrict certain tapes to certain drives for a variety of reasons. Administrative constraints can further limit activity based on volume set, I/O type, etc.
Once the capability matrix is generated, data movers claim work based upon the system state as captured by the last scheduler run.
- Work is apportioned to movers based upon the notion of a "job set". A job set describes the set of tape jobs that share: request type (read aka jget or write aka jput), volume set, volume serial.
- Job sets are assigned priorities based upon the priorities of their constituent jobs, and augmented with dynamic properties.
- A data mover chooses the highest priority job set that matches its capability.
- Having chosen a job set, the data mover will retain an affinity for it until one of the following happens:
- No more jobs exist in the set (i.e. no new jobs of matching type,volume set, volume serial)
- It discovers that some other job set has higher priority.
- The data mover grabs enough jobs to comprise a large chunk of work. It will process tape I/O or yield after about 30 minutes.
Job priorities:
- Priorities are integers with smaller values indicating higher priority
- The priority of each job is comprised of a base value and administrative overrides
- The base value of a jput job = 10
- The base value of a jget job = 20
- The base value is modified +- 3 based upon the job's associated user, volume set, and accounting category nudges, which are configured using administrative tools.
Dynamic job set priority adjustments:
- Each (job set X user) gains a priority nudge that is based upon the amount of time the oldest unfulfilled request has been outstanding. Specifically, the priority is nudged
- -log2(ceil(number of 15 minute intervals since the request was made))
- Each (job set X user) loses a priority nudge that is based upon the amount of time (expressed in tape-minutes) that drives have been devoted to doing their I/O since the oldest unfulfilled request was made. Specifically, the priority is nudged
- log2(ceil(number of 15 tape-minutes since the request was made))
- Each (user x volume set) is docked a nudge for each tape drive that it is currently occupying
The effect of these two adjustments is to elevate waiting jobs asymptotically, tempered by a factor of resource utilization.