Bursty and streaming data

1. Bursty data

Data that is transferred or transmitted in short, uneven spurts. LAN traffic is typically bursty. Contrast with streaming data.

1.1 Burstiness

Burstiness is a characteristic of communications involving data that is transmitted intermittently – in bursts – rather than as a continuous stream.

A burst is a transmission involving a large amount of data sent in a short time, usually triggered as a result of some threshold being reached. When bursts are a repeated, dependable behavior in some type of transmission, the subject in question is said to exhibit burstiness.

The tendency for communications to be bursty is an important factor in the design considerations of the hardware manufacturers make for these communications. For example, a large supply of fast RAM can replaced by lower cost memory with a cache, saving money while potentially having little to no effect on the user experience.

Burstiness is sometimes caused by the nature of the data being communicated. Ethernet, for example, is inherently bursty. Computer architectures that rely more heavily on cache than sustainable bandwidth are typically bursty as well, showing much higher bandwidth until the cache is exhausted and data must be retrieved from sources outside the hardware where the chip resides. Mp3 players have often combined slower storage with a faster cache this way. This type of architecture can prove ineffective if cache memory often fails to contain the desired information.

2. Streaming data

Data that is transmitted and processed in a continuous flow, such as digital audio and video. See audio streaming, video streaming and digital media hub.

Data streaming is the continuous transfer of data at a steady, high-speed rate. To facilitate the need for real-time analytics from disparate data …

2.1 Data streaming

Data streaming is the continuous transfer of data at a steady, high-speed rate. Although the concept of data streaming is not new, its practical applications are a relatively recent development. This is because in the early years of the world wide web, internet connectivity was not always reliable and bandwidth limitations often prevented streaming data to arrive at its destination in an unbroken sequence. Developers created buffers to allow data streams to catch up, but the resulting jitter caused the user experience to be so poor that most consumers preferred to download content rather than stream it.

Today, with the advent of broadband internet, cloud computing and the internet of things (IoT), there is an increased interest in analyzing the data from streaming sources to make data-driven decisions in real time. To facilitate the need for real-time analytics from disparate data sources, many companies have replaced traditional batch processing with streaming data architectures that can accommodate batch processing. In batch processing, newly arriving data elements are collected in a group and the entire group is processed at some future time. In contrast, a streaming data architecture processes data in motion and an ETL batch is treated as just one more event in a continuous stream of events.

Minimum recommended download speeds for viewing streaming data
In order to get a reasonable estimate of bandwidth (sometimes referred to as throughput), experts suggest that three or more different test sites like Fast.com be used, and each test be conducted several times to ensure an accurate read.

• YouTube TV: 13 megabits per second (Mbps) to reliably stream high definition (HD) video
• Netflix: 25 Mbps recommended for 4k Ultra High Definition streaming
• DirecTV Now: 25 Mbps for households that maintain internet use on multiple devices
• PlayStation Vue: At least 20 Mbps download speed to ensure a consistent stream
• Hulu: 25 Mbps for Ultra HD quality
• Amazon Prime Video: 3.5 Mbps for HD videos and 15 Mbps for 4k streaming

Data streaming and big data
To benefit from data streaming at the enterprise level, businesses with streaming architectures require powerful analytics tools for ingesting and processing information. Popular enterprise tools for working with data streams include:

Amazon Kinesis Firehose - an Amazon Web Service (AWS) for processing big data in real time. Kinesis is capable of processing hundreds of terabytes per hour from high volumes of streaming data from sources such as operating logs, financial transactions and social media feeds.

Apache Flink - a distributed data processing platform for use in big data applications, primarily involving analysis of data stored in Hadoop clusters. Flink handles both batch and stream processing jobs, with data streaming the default implementation and batch jobs running as special-case versions of streaming applications.

https://www.pcmag.com/encyclopedia/term/bursty
https://www.pcmag.com/encyclopedia/term/streaming-data
https://whatis.techtarget.com/definition/burstiness
https://searchnetworking.techtarget.com/definition/data-streaming