mysql到redshift_MySQL to Redshift is simplified

Common Problem Area:

MySQL is a popular open source RDBMS. So many companies starts with MySql as backend database solution. Typical scenario is, business starts with small, initial data is not big, so MySql is good enough. When businesses grow and keep growing, problem stated with the need of optimization, scale-up, scale-out then what?

What if data is too BIG? Problem starts with streaming data, when data is continuously filling too fast. Business needs to migrate more data, require to run analysis on full historical data.

Requirement of a Big Database:

Many companies are offering Big database to analyse Big dataset. Amazon is proposing Redshift as cloud based Data Warehousing solution. Greenplum is another alternative. Yes, there are few more.

Transfer data in Warehousing

To analyse Big dataset, data needed to be transferred to Warehousing solution. Technically this part is simple, off-course you need time depending on volume of data, right? When the Change Data Capture (CDC) is the requirement, then challenge started from vendor specific log analysis, error free parser, performance, security, schema migration, can’t break and many more. Even custom requirements are added on top of all standard supports, such as:

I want mark as deleted even though we issue delete query to source database.

Need SQL Console to compare.

What happens when DDL changes are made to tables being captured?

And many more to add…

Again more challenge started in implementation steps, which is all known.

We Need a Simple Solution

Wait, please stop all the buzzwords, we need a simple solution which works. Expectation is simple, need initial data snapshots with standard CDC, shouldn’t be big deal.

A free solution can make your life as easy as drinking water. Not the cluster one, Single node instance is free to use. Yes, it has some constrains like “Data can’t be more then 5 TB”. When you expectation is a simple, workable solution to move data to redshift with no break. I would recommend this solution to try with free single node instance. Thanks to Athena Software Associates Ltd for their good work and offering single node instance free to use, it solves simple problems.