The Unity C# Job System is designed to allow users to write multithreaded code that interacts well with the rest of the Unity engine, and makes it easier to write correct code.
Writing multithreaded code can provide great performance benefits, including significant gains in frame rate, and improved battery life for mobile devices.
An important aspect of the C# Job System is that it integrates with what the engine uses internally (Unity’s native job system). This means that user written code and the engine will share worker threads to avoid creating more threads than CPU cores; which would cause contention for CPU resources.
Multithreading is a type of programming that takes advantage of the fact that a CPU can process multiple threads at the same time. Instead of coded tasks or instructions executing one after another, they execute simultaneously.
A main thread runs at the start of a program by default. Then the main thread creates new threads (often called worker threads) based on the code. The worker threads then run in parallel to one another and usually synchronize their results with the main thread once completed.
This approach to multithreading works well if you have a few tasks that run for a long time. Game development code usually contains multiple small tasks and instructions to execute at once. If you create a thread for each one, you can end up with many threads, each with a short lifetime. This can push the limits of the processing capacity of your CPU and operating system.
You can mitigate the issue of thread lifetime by having a pool of threads, but even if you do, you will have a large amount of threads alive at the same time. Having more threads than CPU cores leads to the threads contending with each other for CPU resources, with frequent context switches as a result.
Context switching is the process of saving the state of a thread part way through execution, then working on another thread, and then reconstructing the first thread later on to continue processing it. Context switching is resource-intensive, so it’s important to avoid the need for it wherever possible.
A job system manages multithreaded code in a different way. Instead of systems creating threads they create something called jobs. A job receives parameters and operates on data, similar to how a method call behaves. Jobs should be fairly small, and do one specific task.
A job system puts jobs into a queue to execute. Worker threads in a job system take items from the job queue and execute them. A job system usually has one worker thread per logical CPU core, to avoid context switching.
If each job is self contained this is all you need. However, this is unlikely in complex systems like those required for game development. So what you usually end up with is one job preparing the data for the next job. To manage this, jobs are aware of and support dependencies.
If job A has a dependency on job B, the job system ensures that job A does not start executing until job B is complete.
When writing multithreaded code there is always a risk for race conditions. A race condition occurs when the output of one operation depends on the timing of another operation outside of its control.
A race condition is not always a bug, but it is always a source of indeterministic behaviour. When a race condition does cause a bug, it can be difficult to find the source of the problem because it depends on timing. This means you can only recreate the issue on rare occasions, and debugging it can cause the problem to disappear; because breakpoints and logging can change the timing too. To a large extent, this is what produces the largest challenge in writing multithreaded code.
To make it easier to write multithreaded code in the C# Job System there is a safety system build in. The safety system detects all potential race conditions and protects you from the bugs they can cause.
The main way the C# Job System solves this is to send each job a copy of the data it needs to operate on, rather than a reference to the data in the main thread. This isolates the data, which eliminates the race condition.
The way the C# Job System copies data means that a job can only access blittable data types (which do not require conversion when passed between managed and native code).
The C# Job System can copy blittable types with memcpy and transfer the data between the managed and native parts of Unity. It uses memcpy to put it into native memory on Schedule and gives the managed side access to that copy on Execute. This can be limiting, because you cannot return a result from the job.
There is one exception to the rule of copying data. That exception is NativeContainers.
A NativeContainer is a managed value type that provides a relatively safe C# wrapper for native memory. When used with the C# Job System, it allows a job to access shared data on the main thread rather than working with a copy of it.
Unity ships with a set of NativeContainers: NativeArray, NativeList, NativeHashMap, and NativeQueue.
Note: Only
NativeArrayis available without the ECS package.
You can also manipulate NativeArrays with NativeSlice to get a subset of the NativeArray from a certain position to a certain length.
All NativeContainers have the safety system built in, which tracks all NativeContainers, and what is reading and writing to them.
For example, if two scheduled jobs are writing to the same NativeArray, the safety system throws an exception with a clear error message that explains why and how to solve the problem. In this case, you can always schedule a job with a dependency so that the first job can write to the NativeContainer, and once it has finished executing, the next job can safely read and write to that NativeContainer as well.
The same read and write restrictions apply when accessing the data from the main thread. The safety system allows many jobs to read from the same data in parallel.
Some NativeContainers also have special rules for allowing safe and deterministic write access from ParallelFor jobs. As an example, the method NativeHashMap.Concurrent lets you add items in parallel from IJobParallelFor.
Note: There is no protection against accessing static data from within a job. Accessing static data circumvents all safety systems and can crash Unity. For more information, see Troubleshooting.
There are three types of Allocators that handle NativeContainer memory allocation and release. You will need to specify one of these when instantiating a NativeContainer.
- Allocator.Temp has the fastest allocation. It is intended for allocations with a lifespan of 1 frame or fewer and is not thread-safe.
TempNativeContainerallocations should call theDisposemethod before you return from the function call. - Allocator.TempJob is a slower allocation than
Temp, but is faster thanPersistent. It is intended for allocations with a lifespan of 4 frames or fewer and is thread-safe. Most short jobs use thisNativeContainerallocation type. - Allocator.Persistent is the slowest allocation, but lasts throughout the application lifetime. It is a wrapper for a direct call to malloc. Longer jobs may use this
NativeContainerallocation type.
For example:
NativeArray<float> result = new NativeArray<float>(1, Allocator.Temp);To schedule a job you need to implement the IJob interface. This allows you to schedule a single job that runs in parallel to other jobs and the main thread. To do this, create an instance of your struct, populate it with data and call the Schedule method. Calling Schedule will put the job into the job queue to be executed at the appropriate time.
When you schedule a job, you will get back a JobHandle you can use in your code as a dependency for other jobs. Otherwise, you can force your code to wait in the main thread for your job to finish executing by calling the method Complete on the JobHandle; at which point you know your code can safely access the NativeContainers on the main thread again.
Note: Jobs do not start executing immediately when you schedule them, unless you state that you are waiting for them in the main thread by calling the method
JobHandle.Complete. This flushes them from the memory cache and starts the process of execution. WithoutJobHandle.Complete, you need to explicitly flush the batch by calling the staticJobHandle.ScheduleBatchedJobsmethod.
Job code:
// Job adding two floating point values together
public struct MyJob : IJob
{
public float a;
public float b;
NativeArray<float> result;
public void Execute()
{
result[0] = a + b;
}
}Main thread code:
// Create a native array of a single float to store the result in. This example will wait for the job to complete, which means we can use Allocator.Temp
NativeArray<float> result = new NativeArray<float>(1, Allocator.Temp);
// Setup the job data
MyJob jobData = new MyJob();
jobData.a = 10;
jobData.b = 10;
jobData.result = result;
// Schedule the job
JobHandle handle = jobData.Schedule();
// Wait for the job to complete
handle.Complete();
// All copies of the NativeArray point to the same memory, we can access the result in "our" copy of the NativeArray
float aPlusB = result[0];
// Free the memory allocated by the result array
result.Dispose();Job code:
public struct AddOneJob : IJob
{
public NativeArray<float> result;
public void Execute()
{
result[0] = result[0] + 1;
}
}Main thread code:
NativeArray<float> result = new NativeArray<float>(1, Allocator.Temp);
// Setup the job data
MyJob jobData = new MyJob();
jobData.a = 10;
jobData.b = 10;
jobData.result = result;
// Schedule the job
JobHandle firstHandle = jobData.Schedule();
AddOneJob incJobData = new AddOneJob();
incJobData.result = result;
JobHandle handle = incJobData.Schedule(firstHandle);
// Wait for the job to complete
handle.Complete();
// All copies of the NativeArray point to the same memory, we can access the result in "our" copy of the NativeArray
float aPlusB = result[0];
// Free the memory allocated by the result array
result.Dispose();When scheduling jobs, there can only be one job doing one task. In a game, it is common to want to perform the same operation on a large number of datapoints. These are called SIMD operations. To handle this, there is a separate job type called IJobParallelFor.
Note: A ParallelFor job is a collective term in Unity for any job that implements the
IJobParallelForinterface.
A ParallelFor job uses a NativeArray as its data source and runs across multiple cores. IJobParallelFor behaves like IJob, but instead of getting a single Execute callback, you get one Execute callback per item in the NativeArray. The system does not actually schedule one job per item, it schedules up to one job per CPU core and redistributes the workload. The system deals with this internally.
When scheduling ParallelForJobs you must specify the length of the NativeArray you are splitting, since the system cannot know which NativeArray you want to use as primary if there are several in the struct. You also need to specify a batch count. The batch count controls how many jobs you get, and how fine-grained the redistribution of work between threads is.
Having a low batch count, such as 1, gives you a more even distribution of work between threads. It does come with some overhead, so sometimes it is better to increase the batch count. Starting at 1 and increasing the batch count until there are negligible performance gains is a valid strategy.
Job code:
// Job adding two floating point values together
public struct MyParallelJob : IJobParallelFor
{
[ReadOnly]
public NativeArray<float> a;
[ReadOnly]
public NativeArray<float> b;
public NativeArray<float> result;
public void Execute(int i)
{
result[i] = a[i] + b[i];
}
}Main thread code:
var jobData = new MyParallelJob();
jobData.a = new NativeArray<float>(new float[] { 1, 2, 3 }, Allocator.TempJob);
jobData.b = new NativeArray<float>(new float[] { 6, 7, 8 }, Allocator.TempJob);
jobData.result = new NativeArray<float>(3, Allocator.TempJob);
// Schedule the job with one Execute per index in the results array and only 1 item per processing batch
JobHandle handle = jobData.Schedule(jobData.result.Length, 1);
// Wait for the job to complete
handle.Complete();
jobData.a.Dispose();
jobData.b.Dispose();
jobData.result.Dispose();When using the C# job system, make sure you adhere to the following:
Accessing static data from a job circumvents all safety systems. If you access the wrong data, you might crash Unity, often in unexpected ways. (Accessing MonoBehaviour can, for example, cause crashes on domain reloads). Because of this risk, future versions of Unity will prevent global variable access from jobs using static analysis, so note that if you do access static data inside a job, you should expect your code to break in future versions of Unity.
When you want your jobs to start, you need to flush the scheduled batch with JobHandle.ScheduleBatchedJobs. Not doing this delays the scheduling until another job waits for the result.
Due to the lack of ref returns, it is not possible to directly change the content of a NativeArray. nativeArray[0]++; is the same as writing var temp = nativeArray[0]; temp++; which does not update the value in the nativeArray. (Unity is working on C# 7.0 support, which will add ref returns and solve this.)
Tracing data ownership requires dependencies to complete before the main thread can use them again. This means that it is not enough to check JobHandle.IsDone. You must call the method JobHandle.Complete to regain ownership of the NativeContainers to the main thread. Calling Complete also cleans up the state in the jobs debugger. Not doing so introduces a memory leak. This also applies if you schedule new jobs every frame that have a dependency on the previous frame's job.