VoIP Checklist

Here's my notes on building a comprehensive VoIP UX (inherited from ye olde TNG days):

P0:

•  Ensure "early hangup" works from both caller & callee side
•  Ensure "late hangup" works from both caller & callee side (i.e. clearing the call after being answered)
•  Ensure multiple consecutive calls work after the various different termination scenarios
•  Ensure that termination reasons are correctly displayed in the UI (e.g. "answered elsewhere", "lost media", "lost signalling", "cancelled", "hung up", "other error")
  answered elsewhere : a toast is displayed.
•  Ensure inbound calls are picked up when the app is launched from cold
•  Ensure inbound calls are picked up when the app is foregrounded after being backgrounded
•  Ensure that multiple devices ring simultaneously for inbound calls
•  Ensure that if the call is answered on another device, we tell the user what the error was
•  Ensure that if the call is rejected by another device, we tell the user that's what happened
•  Ensure that devices which are busy on other calls do not cause inbound calls to be globally rejected (see vector-im/riot-web#1964)
•  Ensure that we don't interact badly with rate limiting (i.e. we correctly retry VoIP signalling like m.call.candidates if we get a 429 'too many requests' response back)
•  Ensure that we don't try to re-answer stale m.call.invite events (m.call.invite and m.call.hangupmatching the same call_id in the same sync resp, so first m.call.invite must be discarded)
•  Ensure 1:1 call is made for 1:1 joined members (invited or banned users do not count)

P1:

•  Need local voice mute button
•  Need local video mute button
•  Need ability to flip between rear & front facing camera
•  Need ability to toggle speakerphone on & off
•  When on video call, default to speakerphone being on.
•  When on voice call, default to speakerphone being off.
•  Ensure that the right microphone is used when on speakerphone (some handsets have a dedicated microphone used when on speakerphone to avoid feedback)
•  When a video call (and not a voice call) is foregrounded, wedge the backlight on.
•  When a voice call (and not a video call) is foregrounded and not on speakerphone, turn the backlight off when the proximity sensor fires (i.e. the phone is brought to the ear).
•  When physical headset is plugged in, switch to headset.
•  Only pause background music when actually on a call
•  Play 'ringtone' and vibrate when the callee receives a call.
•  Ensure "glare" is handled gracefully if caller & callee try to call each other at the same time. Some tiebreaker (e.g. the user with the lexographically lower user ID) should assume the role of caller and the call should proceed.
•  Ensure there is no echo when on speakerphone (as heard by the remote device from a different room)
•  Ensure lipsync is correct
•  Ensure bandwidth/framerate/latency recovers out of going out of reception or being on a bad network
•  Ensure trickle ICE works
•  Ensure TURN works over UDP
•  Ensure TURN works over TCP (port 3478)
•  Ensure TURN works over TCP+SSL (port 443)
•  Ensure permission prompts for microphone and camera are handled well on first use and don't disrupt the call setup (especially if the call setup is happening in the background)
•  Ensure if user cancels permissions for microphone and camera the call still continues as well as it can (and that the user has the ability to easily uncancel the permissions in future)

P2:

•  Ensure that incoming GSM calls put the call on hold
•  Ensure that ongoing calls are taken off hold when the app is subsequently resumed
•  Need indicator to say whether the remote user is currently on hold (especially if this is due to them receiving a GSM call)
•  Need indicator to say whether the remote user has muted voice or video (rather than the call crapping out)
•  Need network quality warning, so that if the call quality is having problems the user understand it's the network's fault rather than the app's fault.
•  Set default audio playback volume (or warn if it is at zero), otherwise users think there's audio and don't realise they have to turn up the volume control.
•  Play 'pushing' tone when waiting for the other user to 'respond to your push' (i.e. return an m.call.ringing or whatever)
•  Play 'ring back tone' when the other user's client(s) is actually ringing. (This is different to a ringtone played for inbound calls!)
•  Show inbound call screen on lockscreen if at all possible.
•  Ensure "back to app" works and video & audio is still captured and audio is still played back when the app is foregrounded but the call UI is backgrounded
•  Ensure that both voice and video calls continue when the app is backgrounded (although the OS will typically disables video capture)
•  Ensure that the phone can be locked and screen turned off (e.g. by pressing power button on iOS) whilst on a call and still have audio continue.
•  Ensure that after N minutes of the phone being locked and screen turned off the call continues and isn't cancelled as a powersaving measure.

P3:

•  Prompt to optionally degrade to voip if insufficient network to support video
•  After N seconds of total media failure, abort call and give option to call back
•  Don't need an explicit "hold" button though as we don't have an explicit line 1/line 2 thing going on (yet) other than when something happens to the call (e.g. GSM interruption) to interrupt voice and video capture.
•  Ensure that other interruptions to the call (e.g. backgrounding it and capturing voice or video with another app) put the call on hold.
•  Ensure that secondary noise-cancelling microphones are correctly used for both speakerphone and non-speakerphone calls. (this is probably best done by googling to see if we're using the right API)
•  Support bluetooth headset
•  Ensure "back to app" button works when an outbound call is being setup (i.e. the call media sets up properly despite connecting in the background)
•  Ensure backgrounded calls work when the app is backgrounded during call setup
•  Ensure that if the OS disables video capture due to backgrounding, the mute is propagated to the receiver
•  Ensure there is no echo when switching speakerphone on and off (may require explicitly telling the VoIP stack when this happens so it knows to recalibrate the echo cancellation filter)
•  Ensure ICE candidate discovery is not too aggressive

In future:

•  Support group calling as per vector-web.
•  Support upgrading voip to video call
•  Support downgrading video to voip call
•  Support upgrading voip/video call to conference
•  Support downgrading voip/video call from conference
•  Support receiving WebRTC screenshare screens
•  Ensure calls survive switching networks
•  Ensure video is played in a thumbnail when "back to app" (like youtube)
•  Prompt for user feedback after calls (rate from 1-5 stars)
•  Report media logs for calls (i.e. phone home with the webrtc.internals graphs, or the TNG equivalent)
•  Hook SRTP DTLS fingerprints into our E2E encryption device keys(!) (for @richvdh)

ref:https://github.com/vector-im/riot-android/issues/351