aws rds同步_将数据从Python同步到AWS RDS

aws rds同步

什么是Amazon Web Services？ (What is Amazon Web Services?)

Amazon Web Services is the cloud platform provided by the Amazon company. It is one of the most frequently emphasized / sought-after platforms in job postings such as data scientist and data engineer. In today’s world, cloud platforms are becoming more and more important as big data grows day by day. In addition to flexibility, speed and practicality, cloud platforms like AWS are built on the philosophy of spending as much as you use with their scalable cost. Many startups, world giants, government agencies use AWS.

亚马逊网络服务是亚马逊公司提供的云平台 。 它是职位发布中最经常强调/最受欢迎的平台之一，例如数据科学家和数据工程师。 在当今世界，随着大数据的日趋增长，云平台变得越来越重要。 除了灵活性，速度和实用性之外，AWS等云平台还建立在可扩展成本的基础上，使您花费尽可能多的费用。 许多创业公司，世界巨头，政府机构都在使用AWS。

什么是AWS RDS(关系数据库服务)？ (What is AWS RDS (Relational Database Service)?)

AWS RDS is the service for creating a traditional database service on the AWS platform. It can be installed very quickly and stands up immediately. You pay as much as you use within the scope of the need. You can resize according to the increasing data size and your need for speed.

AWS RDS是用于在AWS平台上创建传统数据库服务的服务。 它可以快速安装并立即站起来。 您可以根据需要支付尽可能多的费用。 您可以根据不断增长的数据大小和对速度的需求来调整大小 。

If you install the RDS system on AWS, you don’t have to deal with hardware. You do not spend much time for database setup, and operations such as backup are done automatically. Often you just need to play with a few parameters and powers.

如果在AWS上安装RDS系统，则无需处理硬件。 您无需花费太多时间进行数据库设置，并且备份之类的操作是自动完成的。 通常，您只需要玩几个参数和功能。

AWS RDS supports AWS Aurora, PostgreSQL, MySQL, MariaDB, ORACLE, Microsoft SQL database infrastructures.

AWS RDS支持AWS Aurora，PostgreSQL，MySQL，MariaDB，ORACLE，Microsoft SQL数据库基础架构。

Photo by Caspar Camille Rubin on Unsplash

Caspar Camille Rubin在 Unsplash上 拍摄的照片

Let’s Start Creating Database on AWS RDS!

让我们开始在AWS RDS上创建数据库！

1- First we open an AWS account.

1-首先，我们开设一个AWS账户。

We need to enter the AWS homepage to register with AWS. After entering this page, we proceed by saying create an AWS account. In all steps, we must enter our information and especially the e-mail address correctly. Then we need to enter our payment information. Some services of AWS are free to use if you do not exceed the limits. If you have entered payment information, make sure you do not exceed these limits. AWS charges a $ 1 fee to verify your payment account, then returns it.

我们需要进入AWS主页以注册到AWS。 进入此页面后，我们说创建一个AWS账户。 在所有步骤中，我们必须正确输入我们的信息，尤其是电子邮件地址。 然后，我们需要输入付款信息。 如果您未超出限制，则可以免费使用 AWS的某些服务。 如果您输入了付款信息，请确保您没有超过这些限制。 AWS会收取1美​​元的费用来验证您的付款帐户，然后将其退回。

For detailed information, you can follow the steps to create an AWS account.

有关详细信息，您可以按照以下步骤创建AWS账户。

2- Let’s enter to RDS and create the Database.

2-让我们进入RDS并创建数据库。

After creating an AWS account, we enter RDS.

创建一个AWS账户后，我们输入RDS。

We click on the Databases option on the left side of the console. We take the first step to create DB by saying Create database on the page that opens.

我们单击控制台左侧的“数据库”选项。 在打开的页面上说创建数据库 ，我们迈出了创建数据库的第一步。

We need to select the database infrastructure on the page that opens. In this example, we will proceed through MySQL.

我们需要在打开的页面上选择数据库基础结构。 在此示例中，我们将继续进行MySQL 。

For our type of operation, we’re choosing the Free tier for now.

对于我们的操作类型，我们现在选择“免费”套餐。

We need to determine the database name, user name, and password to be created from the Settings section. I created a db called the experiment for the example and created a user named admincv.

我们需要从“设置”部分确定要创建的数据库名称，用户名和密码。 我为示例创建了一个名为实验的数据库，并创建了一个名为admincv的用户。

Amazon provides some standard features for the database to be created. If you are doing this study for trial purposes, it would be better not to turn on Storage autoscaling feature in order not to charge. If you use it within the limits set by Amazon, there will be no charge.

Amazon为要创建的数据库提供了一些标准功能。 如果您出于试用目的进行这项研究，则最好不要打开存储自动扩展功能以免收费。 如果您在Amazon设置的限制内使用它，则不会收费。

After a little wait, the system will make the database available. It will then appear as “Available” in the Status field.

稍等一会，系统将使数据库可用。 然后，它将在“状态”字段中显示为“ 可用 ”。

Now the database has occurred.

现在数据库已经出现。

3- Let’s enter the database we created.

3-让我们输入我们创建的数据库。

Here, the endpoint & port information in the Connectivity & Security area is important and it is useful to take note. This information will then be required so that we can access DB from Python.

在此，“连接性和安全性”区域中的端点和端口信息很重要，请注意。 然后将需要此信息，以便我们可以从Python访问DB。

Now Let’s Switch to Python and Connect to the Database!

现在，让我们切换到Python并连接到数据库！

1- We download our libraries.

1-我们下载我们的库。

import pandas as pd
import pymysql
from sqlalchemy import create_engine

2- We enter our connection information.

2-我们输入连接信息。

We enter the above endpoint link in the host area. We need to select 3306 as a port. The username is “cvadmin” that we have previously created. The password is that you create for this user to connect. Our DB name was a “deneme”. After assigned this to the database variable, we completed all the operations.

我们在主持人区域中输入上述端点链接。 我们需要选择3306作为端口。 用户名是我们先前创建的“ cvadmin”。 密码是您为该用户创建的连接密码。 我们的数据库名称是“ deneme”。 将其分配给数据库变量后，我们完成了所有操作。

host=’deneme.cykgvlpxbgsw.us-east-2.rds.amazonaws.com’
port=int(3306)
user=”cvadmin”
passw=”yourpassw”
database=”deneme”

3- We create an engine by collecting the connection information.

3-我们通过收集连接信息来创建引擎。

Fields such as user, passw in the following code feed on the variables that we created in the previous step.

以下代码中的诸如user，passw之类的字段以我们在上一步中创建的变量为食。

mydb = create_engine(‘mysql+pymysql://’ + user + ‘:’ + passw + ‘@’ + host + ‘:’ + str(port) + ‘/’ + database , echo=False)

4- We print the data we have on the database.

4-我们打印数据库中的数据。

We create a table with the name = “hisse” command and we can now print the data we have. stockshistoricdata, a dataframe I have. It gives daily prices of shares in Bist100. You can also use another DF.

我们使用name =“ hisse”命令创建一个表， 现在可以打印已有的数据 。 stockshistoricdata，我有一个数据框。 它给出Bist100中股票的每日价格。 您也可以使用另一个DF。

stockshistoricdata.to_sql(name=”hisse”, con=mydb, if_exists = ‘replace’, index=False)

Let’s check if the data is printed on the table via DBeaver!

让我们检查是否通过DBeaver将数据打印在表上！

Our data is now in the “hisse” table.

现在，我们的数据在“ hisse”表中。

Conclusion

结论

Why would you want to press a data in Python to a database on AWS in a data science project? You’re right to ask that question. If you are working with big data, if the data is on a daily or weekly new data, and your python code works for minutes or hours, these situations can be a waste of time.

您为什么要在数据科学项目中将Python中的数据按入AWS上的数据库？ 你问这个问题是对的。 如果您正在使用大数据 ，如果数据是每天或每周的新数据 ，并且您的python代码工作了几分钟或几小时 ，那么这些情况可能会浪费时间。

To avoid this situation, you create a script in Python, this new data prints this table every day. It does not touch old data. You can connect to the “hisse” table and work on the accumulated data at any time. You can save time and CPU by printing one-day stock data instead of the effort required when creating all data from scratch. In addition, more than one person can work on the same table.

为了避免这种情况，您可以使用Python创建脚本，此新数据每天都会打印此表。 它不涉及旧数据。 您可以随时连接到“ hisse”表并处理累积的数据。 您可以通过打印一天的库存数据来节省时间和CPU ，而无需从头开始创建所有数据。 此外，同一桌子上可以有多个人一起工作。

I hope my writing has been useful. Have a pleasant reading!

希望我的写作对您有所帮助。 阅读愉快！

Resources;

资源；

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Tutorials.WebServerDB.CreateDBIns

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Tutorials.WebServerDB.CreateDBIns

https://aws.amazon.com/tr/premiumsupport/knowledge-center/create-and-activate-aws-account/?nc1=h_ls

https://aws.amazon.com/tr/premiumsupport/knowledge-center/create-and-activate-aws-account/?nc1=h_ls

My Linkedin Profile;

我的Linkedin个人资料；

https://www.linkedin.com/in/onur-okyol-ba253b72/

https://www.linkedin.com/in/onur-okyol-ba253b72/

翻译自: https://towardsdatascience.com/using-aws-rds-and-python-together-5718a6878e4c

aws rds同步