{Clean Code} Concurrency

"Objects are abstractions of processing. Threads are abstractions of schedule."  

Why Concurrency?

Decoupling what from when can improve both the throughput and structures of an application. This makes the system easier to understand and offers some powerful ways to separate concerns.

Myths and Misconceptions

1. Concurrency always improves performance.

False, sometimes.

2. Design does not change when writing concurrent programs

No, the design of a concurrent algorithm is different.

3. Understanding concurrency issues is not important when working with a container such as a Web or EJB container.

You'd better know about your resources.

More balanced sound bites

Concurrency incurs some overhead, both in performance as well as writing additional code.

Correct concurrency is complex, even for simple problems.

Concurrency bugs are not usually repeatable, so they are often ignored as one-offs.

Concurrency often requires a fundamental change in desgin strategy.

Challeges

Understant what Just-In-Time Compiler does with generated byte-code, and understand what the Java memory model considers to be atomic.

Possible execution paths are too huge.

Concurrency Defense Principles

SRP

Concurrency design is complex and deserves to be separated from the rest of the code.

Concurrency-related code has its own life cycle of development, change and tuning.

Concurrency-related code has its own challenges, which are different from and more difficult than nonconcurrency-related code.

Recommendation: Keep your Concurrency-related code separate from other code.

Corollary: Limit the Scope of Data

Use synchronized word to protect a critical section in the code that uses the shared object.

Recommendation: Take data encapsulation to heart, severely limit the access of any data that may be shared.

Corollary: Use Copies of Data

If using copies of objects allows the code to avoid synchronizing, the savings in avoiding the intrinsic lock will likely make up for the additional creation and garbage collection overhead.

Corollary: Thread Should Be as Independent as Possible

Recommendation: Attempt to partition data into independent subsets than can be operated on by independent threads, possibly in different processors.

Know Your Library

After 1.5, use the thread-safe collections, executor framework for executing unrelated tasks, nonblocking solutions when possible, and notice several library classes are not thread-safe.

java.util.concurrent

ConcurrentHashMap is better than HashMap in nearly all situations.

ReentrantLock: a lock that can be acquired in one method and released in another.

Semaphore: an implementation of the classic semaphore, a lock with a count.

CountDownLatch: a lock that waits for a number of events before releasing all threads waiting on it. This allows all threads to have a fair chance of starting at about the same time.

Recommendation: Review the classes avaiable to you. Be familiar with java.util.concurrent, java.util.concurrent.atomic, java.util.concurrent.locks.

Know Your Execution Models

Basic concepts:

Producer-Consumer: 

a consumer can read only when the queue is not empty, a producer can write only when the queue is not full.

Readers-Writers:

Balance the needs for both readers and writers to satisfy correct operation, provide reasonable throughput and avoiding starvation. 

Make writers wait until there are no readers, but need to notice the starvation and throughput problems.

Dining Philosophers:

Compete resources.

Recommendation: Learn these basic algorithms and understand their solutions.

Beware Dependencies Between Synchronized Methods

Recommendation: Avoid using more than one method on a shared object.

Client-Based Locking: 

Have the client lock the server before calling the first method and make sure the lock's extent includes code calling the last method.

Server-Based Locking:

Within the server, create a method that locks the server, calls all the methods, and then unlocks. Have the client call the new method.

Adapted Server:

Create an intermediary that performs the locking. This is an example of server-based locking, where the original server cannot be changed.

Keep Synchronized Sections Small

Locks are expensive because they increase the overhead and delays.

Writing Correct Shut-down Code Is Hard

Shutdown gracefully is hard.

Recommendation: Think about shut-down early and get it working early. Review existing codes because this is probably harder than you think.

Testing Threaded Code

Recommendation: 

Write tests that have the potential to expose problems and then run them frequently with differentprogrammatic configurations and system configurations and load. If tests ever fail, track down the failure. Don't ignore a failure just because the tests pass on a subsequent run.

Treat spurious failures as candidate threading issues.

• Get your nonthreaded code working first.

  Do not try to chase down nonthreading bugs and threading bugs at the same time.

• Make your threaded code pluggable.

Enable variable configurations.

• Make your threaded code tunable.
• Run with more threads than processors.
• Run on different platforms.
• Instrument your code to try and force failures.

Adding Object.wait/sleep/yield/priority to force failing pathway emerge.

Add jiggle methods to ferret out errors. ConTest --- from IBM.

Conclusion:

Know the possible reasons: multiple threads operating on shared data, using a common resource pool, boundary cases, shutting-down.

Learn your library so you can use the features to solve your problems.

Keep the amount of shared objects as narrow as possible.

Appendix:

Client/Server Example

Where the time is spent:

1. I/O : 

using a socket, connecting to a database, waiting for virtual memory swapping ..

2. Processor:

numerical calculations, regular expression processing, garbage collection ...

 

If the program is I/O bound, adding threads can help improve the performance.

1. consider heavy cilents 

2. consider server's responsibility (consider abstraction level and responsibilities)

E.g 

We can create ClientScheduler interface, and have different implementations.

Possible Paths of Execution

For N instructions in a sequence, no looping or conditionals and T threads, the total number of possible execution paths is equal to 

( ( N * T ) ! ) / ( ( N ! ) ^ T )

Assignment to any 64-bit value requires 2 32-bit assignments, so set long is not atomic.

Know your library

Executor, Nonblocking solutions (AtomicBoolean/Integer, take advantage of modern processors' Compare and Swap operation.), Nonthread-safe classes (SimpleDateFormat, DataBase connections, containers in java.util --- hashtable, putIfAbsent(), servlets)

Dependencies Between Methods Can Break Concurrent Code

Use server-based locking:

It reduces repeated code in clients.

It allows for better performance, easy to replace server.

It reduces the possibility of error.

It enforces a single policy.

It reduces the scope of the shared variables.

If you don't have access to server's codes, use Adapter pattern.

Deadlock:

4 conditions: mutual exclusion (resources cannot be used by multithreads at same time, or limited), lock and wait, no preemption (cannot get other's resources), circular wait.

In this chapter we talked about concurrent update, and the disciplines of clean synchronization and locking that can prevent it. We talked about how threads can enhance the throughput of an I/O-bound system and showed the clean techniques for achieving such improvements. We talked about deadlock and the disciplines for preventing it in a clean way. Finally, we talked about strategies for exposing concurrent problems by instrumenting your code.