Concurrency

Up to this point, you've been learning about sequential programming. Everything in a program happens one step at a time. 

A large subset of programming problems can be solved using sequential programming. For some problems, however, it becomes convenient or even essential to execute several parts of a program in parallel, so that those portions either appear to be executing concurrently, or if multiple processors are available, actually do execute simultaneously.

Parallel programming can produce great improvements in program execution speed, provide an easier model for designing certain types of programs, or both. However, becoming adept at concurrent programming theory and techniques is a step up from everything you've learned so far in this book,and is an intermediate to advanced topic. This chapter can only serve as an introduction, and you should by no means consider yourself a good concurrent programmer even if you understand this chapter thoroughly.

As you shall see, the real problem with concurrency occurs when tasks that are executing in parallel begin to interfere with each other. This can happen in such a subtle and occasional manner that it's probably fair to say that  concurrency is "arguably deterministic but effectively nondeterministic." That  is, you can make an argument to conclude that it's possible to write concurrent programs that, through care and code inspection, work correctly.In practice, however, it's much easier to write concurrent programs that only appear to work, but given the right conditions, will fail.These conditions may never actually occur, or occur so infrequently that you never see them during testing. In fact, you may not be able to write test code that will generate failure conditions for your concurrent program. The resulting failures will often only occur occasionally, and as a result they appear in the form of customer complaints. This is one of the strongest arguments for studying concurrency: If you ignore it, you're likely to get bitten.

Concurrency thus seems fraught with peril, and if that makes you a bit fearful, this is probably a good thing. Although Java SE5 has made significant improvements in concurrency, there are still no safety nets like compile-time verification or checked exceptions to tell you when you make a mistake. With concurrency, you're on your own, and only by being both suspicious and aggressive can you write multithreaded code in Java that will be reliable.

 

 Alas, if only it were so. Unfortunately, you don't get to choose when threads will appear in your Java programs. Just because you never start a thread yourself doesn't mean you'll be able to avoid writing threaded code. For example, Web systems are one of the most common Java applications, and the basic Web library class, the servlet, is inherently multithreaded—this is essential because Web servers often contain multiple processors, and concurrency is an ideal way to utilize these processors. As simple as a servlet might seem, you must understand concurrency issues in order to use servlets properly. The same goes for graphical user interface programming, as you shall see in the Graphical User Interfaces chapter. Although the Swing and SWT libraries both have mechanisms for thread safety, it's hard to know how 
to use these properly without understanding concurrency.

Java is a multithreaded language, and concurrency issues are present whether you are aware of them or not. As a result, there are many Java programs in use that either just work by accident, or work most of the time and mysteriously break every now and again because of undiscovered concurrency flaws. Sometimes this breakage is benign, but sometimes it means the loss of valuable data, and if you aren't at least aware of  concurrency issues, you may end up assuming the problem is somewhere else rather than in your software. These kinds of issues can also be exposed or amplified if a program is moved to a multiprocessor system. Basically,knowing about concurrency makes you aware that apparently correct programs can exhibit incorrect behavior. 

Concurrent programming is like stepping into a new world and learning a new language, or at least a new set of language concepts. Understanding concurrent programming is on the same order of difficulty as understanding object-oriented programming. If you apply some effort, you can fathom( 看穿) the basic mechanism, but it generally takes deep study and understanding to  develop a true grasp of the subject. The goal of this chapter is to give you a solid foundation in the basics of concurrency so that you can understand the concepts and write reasonable multithreaded programs. Be aware that you can easily become overconfident. If you are writing anything complex, you will need to study dedicated books on the topic. 

The many faces of concurrency

A primary reason why concurrent programming can be confusing is that there is more than one problem to solve using concurrency, and more than one approach to implementing concurrency, and no clean mapping between the two issues (and often a blurring of the lines all around). As a result, you're forced to understand all issues and special cases in order to use concurrency effectively. 

The problems that you solve with concurrency can be roughly classified as "speed" and "design manageability." 

Faster execution 

The speed issue sounds simple at first: If you want a program to run faster,break it into pieces and run each piece on a separate processor. Concurrency is a fundamental tool for multiprocessor programming. Now, with Moore's Law running out of steam (at least for conventional chips), speed improvements are appearing in the form of multicore processors rather than faster chips. To make your programs run faster, you'll have to learn to take advantage of those extra processors, and that's one thing that concurrency gives you.

If you have a multiprocessor machine, multiple tasks can be distributed across those processors, which can dramatically improve throughput. This is often the case with powerful multiprocessor Web servers, which can distribute large numbers of user requests across CPUs in a program that allocates one thread per request.

However, concurrency can often improve the performance of programs running on a single processor.

  

This can sound a bit counterintuitive. If you think about it, a concurrent program running on a single processor should actually have more overhead than if all the parts of the program ran sequentially, because of the added cost of the so-called context switch (changing from one task to another). On the surface, it would appear to be cheaper to run all the parts of the program as a single task and save the cost of context switching. 

A very common example of performance improvements in single-processor systems is event-driven programming. Indeed, one of the most compelling reasons for using concurrency is to produce a responsive user interface. Consider a program that performs some long-running operation and thus ends up ignoring user input and being unresponsive. If you have a "quit" button, you don't want to be forced to poll it in every piece of code you write. This produces awkward code, without any guarantee that a programmer won't forget to perform the check. Without concurrency, the only way to produce a responsive user interface is for all tasks to periodically check for user input. By creating a separate thread of execution to respond to user input, even though this thread will be blocked most of the time, the program guarantees a certain level of responsiveness.

The program needs to continue performing its operations, and at the same time it needs to return control to the user interface so that the program can respond to the user. But a conventional method cannot continue performing its operations and at the same time return control to the rest of the program.In fact, this sounds like an impossibility, as if the CPU must be in two places at once, but this is precisely the illusion that concurrency provides (in the case of multiprocessor systems, this is more than just an illusion). 

One very straightforward way to implement concurrency is at the operating system level, using processes. A process is a self-contained program running within its own address space. A multitasking operating system can run more than one process (program) at a time by periodically switching the CPU from one process to another, while making it look as if each process is chugging along on its own. Processes are very attractive because the operating system usually isolates one process from another so they cannot interfere with each other, which makes programming with processes relatively easy. In contrast,concurrent systems like the one used in Java share resources like memory and I/O, so the fundamental difficulty in writing multithreaded programs is coordinating the use of these resources between different thread-driven tasks, 
so that they cannot be accessed by more than one task at a time. 

Here's a simple example that utilizes operating system processes. While writing a book, I regularly make multiple redundant backup copies of the current state of the book. I make a copy into a local directory, one onto a memory stick, one onto a Zip disk, and one onto a remote FTP site. To automate this process, I wrote a small program (in Python, but the concepts are the same) which zips the book into a file with a version number in the name and then performs the copies. Initially, I performed all the copies sequentially, waiting for each one to complete before starting the next one.But then I realized that each copy operation took a different amount of time depending on the I/O speed of the medium. Since I was using a multitasking operating system, I could start each copy operation as a separate process and let them run in parallel, which speeds up the execution of the entire program.While one process is blocked, another one can be moving forward. 

This is an ideal example of concurrency. Each task executes as a process in its own address space, so there's no possibility of interference between tasks.More importantly, there's no need for the tasks to communicate with each other because they're all completely independent. The operating system minds all the details of ensuring proper file copying. As a result, there's no risk and you get a faster program, effectively for free. 

Some people go so far as to advocate processes as the only reasonable approach to concurrency, but unfortunately there are generally quantity and overhead limitations to processes that prevent their applicability across the concurrency spectrum. 

Some programming languages are designed to isolate concurrent tasks from each other. These are generally called functional languages, where each function call produces no side effects (and so cannot interfere with other functions) and can thus be driven as an independent task. Erlang is one such language, and it includes safe mechanisms for one task to communicate with another. If you find that a portion of your program must make heavy use of concurrency and you are running into excessive problems trying to build that portion, you may want to consider creating that part of your program in a dedicated concurrency language like Erlang. 

Java took the more traditional approach of adding support for threading on top of a sequential language(It could be argued that trying to bolt concurrency onto a sequential language is a doomed approach, but you'll have to draw your own conclusions. ). Instead of forking external processes in a multitasking operating system, threading creates tasks within the single process represented by the executing program. One advantage that this provided was operating system transparency, which was an important design goal for Java. For example, the pre-OSX versions of the Macintosh operating system (a reasonably important target for the first versions of Java) did not 
support multitasking. Unless multithreading had been added to Java, any concurrent Java programs wouldn't have been portable to the Macintosh and similar platforms, thus breaking the "write once/run everywhere" requirement（This requirement was never completely fulfilled and is no longer so loudly touted by Sun. Ironically, one reason that "write once/run everywhere" didn't completely work may have resulted from problems in the threading system—which might actually be fixed in Java 
SE5. ）.

Improving code design 

A program that uses multiple tasks on a single-CPU machine is still just doing one thing at a time, so it must be theoretically possible to write the same program without using any tasks. However, concurrency provides an important organizational benefit: The design of your program can be greatly simplified. Some types of problems, such as simulation, are difficult to solve without support for concurrency. 

Most people have seen at least one form of simulation, as either a computer game or computer-generated animations within movies. Simulations generally involve many interacting elements, each with "a mind of its own."Although you may observe that, on a single-processor machine, each simulation element is being driven forward by that one processor, from a programming standpoint it's much easier to pretend that each simulation element has its own processor and is an independent task. 

A full-fledged simulation may involve a very large number of tasks,corresponding to the fact that each element in a simulation can act independently—this includes doors and rocks, not just elves and wizards.Multithreaded systems often have a relatively small size limit on the number of threads available, sometimes on the order of tens or hundreds. This number may vary outside the control of the program—it may depend on the platform, or in the case of Java, the version of the JVM. In Java, you can 
generally assume that you will not have enough threads available to provide one for each element in a large simulation. 

A typical approach to solving this problem is the use of cooperative multithreading. Java's threading is preemptive, which means that a scheduling mechanism provides time slices for each thread, periodically interrupting a thread and context switching to another thread so that each one is given a reasonable amount of time to drive its task. In a cooperative system, each task voluntarily gives up control, which requires the programmer to consciously insert some kind of yielding statement into each 
task. The advantage to a cooperative system is twofold: Context switching is typically much cheaper than with a preemptive system, and there is theoretically no limit to the number of independent tasks that can be running at once. When you are dealing with a large number of simulation elements,this can be the ideal solution. Note, however, that some cooperative systems are not designed to distribute tasks across processors, which can be very limiting. 

At the other extreme, concurrency is a very useful model—because it's what is actually happening—when you are working with modern messaging systems, which involve many independent computers distributed across a network. In this case, all the processes are running completely independently of each other, and there's not even an opportunity to share resources. However, you must still synchronize the information transfer between processes so that the entire messaging system doesn't lose information or incorporate information at incorrect times. Even if you don't plan to use concurrency very much in your immediate future, it's helpful to understand it just so you can grasp messaging architectures, which are becoming more predominant ways to create distributed systems. 

Concurrency imposes costs, including complexity costs, but these are usually outweighed by improvements in program design, resource balancing, and user convenience. In general, threads enable you to create a more loosely coupled design; otherwise, parts of your code would be forced to pay explicit attention to tasks that would normally be handled by threads. 

Basic threading 

Concurrent programming allows you to partition a program into separate,independently running tasks. Using multithreading, each of these independent tasks (also called subtasks) is driven by a thread of execution. A thread is a single sequential flow of control within a process. A single process can thus have multiple concurrently executing tasks, but you program as if each task has the CPU to itself. An underlying mechanism divides up the CPU time for you, but in general, you don't need to think about it. 

The threading model is a programming convenience to simplify juggling several operations at the same time within a single program: The CPU will pop around and give each task some of its time.Each task has the consciousness of constantly having the CPU to itself, but the CPU's time is being sliced among all the tasks (except when the program is actually running on multiple CPUs). One of the great things about threading is that you are abstracted away from this layer, so your code does not need to know whether it is running on a single CPU or many. Thus, using threads is a way to create transparently scalable programs—if a program is running too slowly, you can 
easily speed it up by adding CPUs to your computer. Multitasking and multithreading tend to be the most reasonable ways to utilize multiprocessor systems. 

Defining tasks 

A thread drives a task, so you need a way to describe that task. This is provided by the Runnable interface. To define a task, simply implement Runnable and write a run( ) method to make the task do your bidding. 

For example, the following LiftOff task displays the countdown before liftoff: 

//: concurrency/LiftOff.java 
// Demonstration of the Runnable interface. 
public class LiftOff implements Runnable {
	protected int countDown = 10; // Default
	private static int taskCount = 0;
	private final int id = taskCount++;

	public LiftOff() {
	}

	public LiftOff(int countDown) {
		this.countDown = countDown;
	}

	public String status() {
		return "#" + id + "(" + (countDown > 0 ? countDown : "Liftoff!")
				+ ") , ";
	}

	public void run() {
		while (countDown-- > 0) {
			System.out.print(status());
			Thread.yield();
		}
	}
} // /: -

The identifier id distinguishes between multiple instances of the task. It is final because it is not expected to change once it is initialized. 

A task's run( ) method usually has some kind of loop that continues until the task is no longer necessary, so you must establish the condition on which to break out of this loop (one option is to simply return from run( )). Often,run( ) is cast in the form of an infinite loop, which means that, barring some factor that causes run( ) to terminate, it will continue forever (later in the chapter you'll see how to safely terminate tasks). 

The call to the static method Thread.yield( ) inside run( ) is a suggestion to the thread scheduler (the part of the Java threading mechanism that moves the CPU from one thread to the next) that says, "I've done the important parts of my cycle and this would be a good time to switch to another task for a while."It's completely optional, but it is used here because it tends to produce more interesting output in these examples: You're more likely to see evidence of tasks being swapped in and out.

In the following example, the task's run( ) is not driven by a separate thread;it is simply called directly in main( ) (actually, this is using a thread: the one that is always allocated for main( )): 

//: concurrency/MainThread.Java 
public class MainThread { 
public static void main(String[] args) { 
	LiftOff launch = new LiftOff(); 
	launch.run(); 
} 
} /* Output: 
#0(9), #0(8), #0(7), #0(6), #0(5), #0(4), #0(3), #0(2), 
#0(1), #0(Liftoff!), 
*///:-

When a class is derived from Runnable, it must have a run( ) method, but that's nothing special—it doesn't produce any innate threading abilities. To achieve threading behavior, you must explicitly attach a task to a thread.

The Thread class 

The traditional way to turn a Runnable object into a working task is to hand it to a Thread constructor. This example shows how to drive a Liftoff object using a Thread: 

//: concurrency/BasicThreads.Java 
// The most basic use of the Thread class. 
public class BasicThreads {
	public static void main(String[] args) {
		Thread t = new Thread(new LiftOff());
		t.start();
		System.out.println("Waiting for Liftoff");
	}
} /*
 * Output: (90% match) Waiting for Liftoff #0(9), #0(8), #0(7), #0(6), #0(5),
 * #0(4), #0(3), #0(2), #0(1), #0(Liftoff !) ,
 */// :-

A Thread constructor only needs a Runnable object. Calling a Thread object's start( ) will perform the necessary initialization for the thread and then call that Runnable's run( ) method to start the task in the new thread. Even though start( ) appears to be making a call to a long-running method,you can see from the output—the "Waiting for LiftOff' message appears before the countdown has completed—that start( ) quickly returns. In effect,you have made a method call to LiftOff.run( ), and that method has not yet finished, but because LiftOff.run( ) is being executed by a different thread,you can still perform other operations in the main( ) thread. (This ability is not restricted to the main( ) thread— any thread can start another thread.)Thus, the program is running two methods at once—main( ) and LiftOff.run( ). run( ) is the code that is executed "simultaneously" with the other threads in a program. 

You can easily add more threads to drive more tasks. Here, you can see how all the tasks run in concert with one another(In this case, a single thread (main( )), is creating all the LiftOff threads. If you have multiple threads creating LiftOff threads, however, it is possible for more than one LiftOff to have the same id. You'll learn why later in this chapter. ):

// : concurrency/MoreBasicThreads.Java 
// Adding more threads. 
public class MoreBasicThreads {
	public static void main(String[] args) {
		for (int i = 0; i < 5; i++)
			new Thread(new LiftOff()).start();
		System.out.println("Waiting for LiftOff");
	}
}

The output shows that the execution of the different tasks is mixed together as the threads are swapped in and out. This swapping is automatically controlled by the thread scheduler. If you have multiple processors on your machine, the thread scheduler will quietly distribute the threads among the processors(This was not true for some of the earliest versions of Java. ).
The output for one run of this program will be different from that of another,because the thread-scheduling mechanism is not deterministic. In fact, you may see dramatic differences in the output of this simple program between one version of the JDK and the next. For example, an earlier JDK didn't time-slice very often, so thread 1 might loop to extinction first, then thread 2 would go through all of its loops, etc. This was virtually the same as calling a routine that would do all the loops at once, except that starting up all those threads is more expensive. Later JDKs seem to produce better time-slicing behavior, so each thread seems to get more regular service. Generally, these kinds of JDK behavioral changes have not been mentioned by Sun, so you cannot plan on any consistent threading behavior. The best approach is to be as conservative as possible while writing threaded code. 

Thread对象的特殊性：

When main( ) creates the Thread objects, it isn't capturing the references for any of them. With an ordinary object, this would make it fair game for garbage collection, but not with a Thread. Each Thread "registers" itself so there is actually a reference to it someplace, and the garbage collector can't clean it up until the task exits its run( ) and dies. You can see from the output that the tasks are indeed running to conclusion, so a thread creates a separate thread of execution that persists after the call to start( ) completes.

Using Executors 

Java SE5 java.util.concurrent Executors simplify concurrent programming by managing Thread objects for you. Executors provide a layer of indirection between a client and the execution of a task; instead of a client executing a task directly, an intermediate object executes the task.Executors allow you to manage the execution of asynchronous tasks without having to explicitly manage the lifecycle of threads. Executors are the preferred method for starting tasks in Java SE5/6. 

We can use an Executor instead of explicitly creating Thread objects in MoreBasicThreads.java. A LiftOff object knows how to run a specific task; like the Command design pattern, it exposes a single method to be executed. An ExecutorService (an Executor with a service lifecycle—e.g., shutdown) knows how to build the appropriate context to execute Runnable objects. In the following example, the CachedThreadPool creates one thread per task. Note that an ExecutorService object is created using a static Executors method which determines the kind of Executor it will be: 

// : concurrency/CachedThreadPool.Java 
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class CachedThreadPool {
	public static void main(String[] args) {
		ExecutorService exec = Executors.newCachedThreadPool();
		for (int i = 0; i < 5; i++)
			exec.execute(new LiftOff());
		exec.shutdown();
	}
}

Very often, a single Executor can be used to create and manage all the tasks in your system. 

The call to shutdown( ) prevents new tasks from being submitted to that Executor. The current thread (in this case, the one driving main( )) will continue to run all tasks submitted before shutdown( ) was called. The program will exit as soon as all the tasks in the Executor finish. 

You can easily replace the CachedThreadPool in the previous example with a different type of Executor. A FixedThreadPool uses a limited set of threads to execute the submitted tasks: 

//: concurrency/FixedThreadPool.Java 
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class FixedThreadPool {
	public static void main(String[] args) {
		// Constructor argument is number of threads:
		ExecutorService exec = Executors.newFixedThreadPool(5);
		for (int i = 0; i < 5; i++)
			exec.execute(new LiftOff());
		exec.shutdown();
	}
}

With the FixedThreadPool, you do expensive thread allocation once, up front, and you thus limit the number of threads. This saves time because you aren't constantly paying for thread creation overhead for every single task.Also, in an event-driven system, event handlers that require threads can be serviced as quickly as you want by simply fetching threads from the pool. You don't overrun the available resources because the FixedThreadPool uses a bounded number of Thread objects. 
Note that in any of the thread pools, existing threads are automatically reused when possible. 

线程池的选择：

Although this book will use CachedThreadPools, consider using FixedThreadPools in production code. A CachedThreadPool will generally create as many threads as it needs during the execution of a program and then will stop creating new threads as it recycles the old ones, so it's a reasonable first choice as an Executor. Only if this approach causes problems do you need to switch to a FixedThreadPool. 

A SingleThreadExecutor is like a FixedThreadPool with a size of one thread(It also offers an important concurrency guarantee that the others do not—no two tasks 
will be called concurrently. This changes the locking requirements for the tasks (you'll learn about locking later in the chapter). ). This is useful for anything you want to run in another thread continually (a long-lived task), such as a task that listens to incoming socket connections. It is also handy for short tasks that you want to run in a thread—for example, small tasks that update a local or remote log, or for an event-dispatching thread. 

If more than one task is submitted to a SingleThreadExecutor, the tasks will be queued and each task will run to completion before the next task is begun, all using the same thread. In the following example, you'll see each task completed, in the order in which it was submitted, before the next one is begun. Thus, a SingleThreadExecutor serializes the tasks that are submitted to it, and maintains its own (hidden) queue of pending tasks. 

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

// : concurrency/SingleThreadExecutor.Java 
public class SingleThreadExecutor {
	public static void main(String[] args) {
		ExecutorService exec = Executors.newSingleThreadExecutor();
		for (int i = 0; i < 5; i++)
			exec.execute(new LiftOff());
		exec.shutdown();
	}
}

As another example, suppose you have a number of threads running tasks that use the file system. You can run these tasks with a SingleThreadExecutor to ensure that only one task at a time is running from any thread. This way, you don't need to deal with synchronizing on the shared resource (and you won't clobber the file system in the meantime).Sometimes a better solution is to synchronize on the resource (which you'll learn about later in this chapter), but a SingleThreadExecutor lets you skip the trouble of getting coordinated properly just to prototype something.By serializing tasks, you can eliminate the need to serialize the objects. 

Producing return values from tasks

A Runnable is a separate task that performs work, but it doesn't return a value. If you want the task to produce a value when it's done, you can implement the Callable interface rather than the Runnable interface.Callable, introduced in Java SE5, is a generic with a type parameter representing the return value from the method call( ) (instead of run( )),and must be invoked using an ExecutorServdce submit( ) method. Here's a simple example: 

//: concurrency/CallableDemo.java 
import java.util.concurrent.*;
import java.util.*;

class TaskWithResult implements Callable<String> {
	private int id;

	public TaskWithResult(int id) {
		this.id = id;
	}

	public String call() {
		return "result of TaskWithResult " + id;
	}
}

public class CallableDemo {
	public static void main(String[] args) {
		ExecutorService exec = Executors.newCachedThreadPool();
		ArrayList<Future<String>> results = new ArrayList<Future<String>>();
		for (int i = 0; i < 10; i++)
			results.add(exec.submit(new TaskWithResult(i)));
		for (Future<String> fs : results)
			try {
				// get( ) blocks until completion:
				System.out.println(fs.get());
			} catch (InterruptedException e) {
				System.out.println(e);
				return;
			} catch (ExecutionException e) {
				System.out.println(e);
			} finally {
				exec.shutdown();
			}
	}
}

The submit( ) method produces a Future object, parameterized for the particular type of result returned by the Callable. You can query the Future with isDone( ) to see if it has completed. When the task is completed and has a result, you can call get( ) to fetch the result. You can simply call get( ) without checking isDone( ), in which case get( ) will block until the result is ready. You can also call get( ) with a timeout, or isDone( ) to see if the task has completed, before trying to call get( ) to fetch the result. 
The overloaded Executors.callable( ) method takes a Runnable and produces a Callable. ExecutorService has some "invoke" methods that run collections of Callable objects.