How can I install the Google Apps Package (Play Store, ...) on my Android device?

Background

Only Android devices that are licensed by the Open Handset Alliance contain the so called Google Apps. Usually people associate apps like Google Maps, Google Play Store, etc. with Android, but because of the open-source nature of Android, manufactures can sell Android devices without Google Apps and therefore avoid to pay the license fee.

Custom ROMs

At first cyanogenmod shipped its ROM with pre-installed Google Apps. In September 2009 the founder of Cyanogen received a cease and desist letter from Google, demanding that the Google Apps not be included in the team's ROMs, since the ROM could be ported to devices which are not licensed to have the Google Apps. In turn, that would bypass the requirement for the devices to be licensed. For a few days, development on the ROM ceased, there were outcries from Android users all over the Internet, and many bloggers suspected Google had just shot itself in the foot.

Eventually, an alternate method was found, whereby Google Apps are not included in the ROM proper but the Google Apps which came with a device would be backed up by the device's custom recovery or another app, and then re-installed onto the device once the new ROM was flashed. This would therefore "Google-ify" one's ROM.

^Source

How to install Google Apps?

Generally speaking:

First download the package corresponding to your version of android. Currently (november 2015), there are several "distributors" of Gapps available. Each typically offers a set of packages per android version, each package with more content. The apps in the packages range from a lite package with the basic apps (~100MB) to all of googles apps(200+MB). The applications themselves are untouched.

If you don't want to or cannot root your device:
1. Flash the Gapps (Google Apps Package) zip package with a custom recovery.

You may first need to flash a custom recovery (like CWM or TWRP) using a PC tool such as Fastboot or Odin/Heimdall. This flashing in turn may require an unlocked bootloader on the phone.

If root is not a problem: (See if you can root the device)

Some stock recoveries will let you "update with adb". Just install adb, then adb command to install is:adb sideload gapps.zip--assuming the package is in the adb folder and is called gapps.zip

You may also install a custom recovery like in the unrooted case. Root typically allows android apps to flash a new recovery (i.e. without a pc).

Install a ROM with the Google Framework, if it exists for your device. Though many ROM developers have followed CyanogenMod's example of separating distribution.

If you flash a Gapps ROM, it may be possible to re-flash with another non-Gapps ROM afterwards while retaining the Gapps. This assumes the android versions are compatible. Most users capable of flashing a ROM will be capable of flashing Gapps separately.