Background
Concurrency Hanlding
- Listener & Worker :
- Listener : to listen and accept user request;
- Worker : to handle and respond to user request;
- Single Process : handle only one request at a given time;
- No Concurrency
- Multi Processes : handle multiple requests at a given time;
- Examples : FTP, Postgres, ancient WebServer (HTTP)
- Limited Concurrency due to Heavy Resource Consumption
- Dedicated memory
- Cost on create and destory processes;
- Multi Threads : handle multiple requests at a given time;
- Improved Concurrency due to Light Resource Consumption
- Shared Memory (however Thread Synchronization)
- Cost on create and descory threads is less than processes;
- Improved Concurrency due to Light Resource Consumption
- Further Optimization : Reduce the cost of create/destory process/threads
- Pooling processes/threads;
- Further Optimization : Reduce context switching cost
- NIO - Selector;
Long Connections
- Result in Long Standing Processes/Threads and result in heavy resource consumption.
- Cost : Threads - 1M; Context Switch;
- Scale Out : use multiple servers;
- Scale Up : reuse threads for multiple requests/connections,
- Scenarios : Thread is not fully utilized, i.e. most time idle - waiting for I/O activities;
- Example : excessive amount of connections, yet less data transferred; ChatApp;
Thread
Declaration
Extend Thread class OR Implement Runnable Interface; And override run() function;
// Thread doing nothing, i.e. empty run()
new Thread().start();
// Thread doing nothing, i.e. empty run()
Thread thread = new Thread();
thread.start();
// Thread doing what is specified in ChileThread.run();
Thread thread = new ChildThread();
thread.start();
// Thread doing what is specified in objImplementRunnable.run();
Thread thread = new Thread(objImplementRunnable);
thread.start();
// No thread, but local (the same thread) method call;
Thread thread = new ChildThread();
thread.run();
Return Value
Race Condition
Cannot rely on the sequence of method invocation to assess the sequence of method completion, because sequence of Thread execution is not controllable.
public class Something {
public void masterThread() {
Thread worker = new WorkerThread();
worker.start();
// some other processing
...
// RACE CONDITION : don't know if WorkerThread has run and completed;
worker.getResult();
}
}
public class WorkerThread() extends Thread {
private int param1;
private int param2;
private int result;
WorkerThread(int x, int y) {
this.param1 = x;
this.param2 = y;
}
@override
public void run() {
result = param1 + param2;
}
public int getResult() {
return result;
}
}
Master to POLL
Polling the status flag in the worker process, retrive the result only after flag is set; Problem is that the POLLING in master consume a lot of CPU power, and WorkerThread may not even get the opportunity to run;
public class Something {
public void masterThread() {
Thread worker = new WorkerThread();
worker.start();
// some other processing
...
// POLLING : check the flag in WorkerThread;
while (true) {
if (worker.getWorkDone()) {
worker.getResult();
break;
}
}
}
}
public class WorkerThread() extends Thread {
private int param1;
private int param2;
private int result;
// add a flag, indicating if work done
private boolean workDone;
WorkerThread(int x, int y) {
this.param1 = x;
this.param2 = y;
workDone = false;
}
@override
public void run() {
result = param1 + param2;
// there might be the issue of code resequencing by JVM;
workDone = true;
}
public int getResult() {
return result;
}
// method allowing master to poll and retrieve work status
public boolean getWorkDone() {
return workDone;
}
}
Worker to CALLBACK
WorkerThread to callback a method exposed by MasterThread to send the result back, at the end of its run() function; This callback exposed by MasterThread can be either static callback or instance callback. In case of instance callback, the reference of the MasterThread has to be passed over to the WorkerThread;
public class Something {
public void masterThread() {
Thread worker = new WorkerThread();
worker.start();
// some other processing
...
}
// Address/method for worker to CALLBACK
public static void callBackAddress(int result) {
...
}
}
public class WorkerThread() extends Thread {
private int param1;
private int param2;
WorkerThread(int x, int y) {
this.param1 = x;
this.param2 = y;
}
@override
public void run() {
int result = param1 + param2;
// CALLBACK : Send the result via CALLBACK
Something.callBackAddress(result);
}
}
Future, Callable & Executor
Introduced in Java5, Future/Callable provide a new approach for Master to get the result from Worker thread, which fundementally relies on CALLBACK mechanism. Future.get() blocks until it gets the result from WorkerThread.
public class Something {
public void masterThread() {
// create a callable task
WorkerThread task = new WorkerThread(x, y);
// create a threadPool
ExecutorService threadPool = Executors.newFixedThreadPool(2);
// execute task using ThreadPool; Use Future to get the RESULT - actually a CALLBACK address;
Future<Integer> result = threadPool.submit(task);
// some other processing
...
// get Result - BLOCKED until result is available;
Integer result = result.get();
}
}
public class WorkerThread() extends Callable<Integer> {
private int param1;
private int param2;
WorkerThread(int x, int y) {
this.param1 = x;
this.param2 = y;
}
@override
public Integer call() {
int result = param1 + param2;
return result;
}
}
Scheduling
Not an issue if I/O heavy, since most likely threads will be blocked for I/O activities. However, if computation heavy, CPU will more likely to be the critical resources, with many threads to compete to get CPU to run their code, which may leads to THREAD STARVATION.
Priority
Java : 0 ~ 10, with 5 being the default; However, not all OS support 11 priorities, for example, Windows only support 7. Java Thread Priority Constants
public static final int MIN_PRIORITY = 1;
public static final int NORM_PRIORITY = 5;
public static final int MAX_PRIORITY = 10;
Preemptive
Java use Preemptive scheduling :
- Preemptive : Scheduler determine if there is another thread that deserve CPU time, if yes, switch to the other one;
- Cooperative : Scheduler wait the current thread to pause and release the CPU. (could cause more starvation);
Pause Condition
Type | Description | Release Obtaind Lock? |
---|---|---|
Block |
| No |
Yield |
| No |
Sleep |
| No |
Join |
| No |
Object.Wait() |
| YES : Lock ob Object No : others |
Finish | run() completes | Yes |
Synchronization
Two threads access the shared resources; Need to ensure their access is in proper sequence.
Approach
- sychronized
- refain from using shared resources
- ThreadLocal
- automatic