When the user wishes to exit all open activities, they should press a button which loads the first Activity that runs when your app starts, in my case "LoginActivity".
Intent intent = new Intent(getApplicationContext(), LoginActivity.class);
intent.setFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP);
intent.putExtra("EXIT", true);
startActivity(intent);
The above code clears all the activities except for LoginActivity. LoginActivity is the first activity that is brought up when the user runs the program. Then put this code inside the LoginActivity's onCreate, to signal when it should self destruct when the 'Exit' message is passed.
if (getIntent().getBooleanExtra("EXIT", false)) {
finish();
}