python课堂讨论_Data Science with Python

Description

This event will show and demo the full pipeline about data science workflow, from data fetching ETL use Loopback API Nodejs to fetch data from openapi, then use Spark to do the I/O, apply data integration and  data processing to make data ready for the modelling, we will go through to set up the pySpark on Docker, then explain Spark operation functions and RDDs for the dataset (use Nodejs Loopback API) to get data from backend. The workshop will cover data infra and architecture design, data processing and integration by using big data framework pySpark, Implement dataset to split into training/validation/test sets and modeling with one supervised learning algorithm.

AGENDA

- Create some input RDDs from external data or parallelize collection in your delivered program

- Lazily transform them to define new RDDs using transforming like filter() or map()

- Ask Spark to cache() any intermediate RDDs that will need to be reused

- Launch actions such as count() and collect () to kick off a parallel computation, which is then optimized and executed by Spark

REQUIREMENTS

•A laptop

•Spark learning resources

•Know some coding basic concepts

ABOUT THE SPEAKER

Chloe is the data analyst in Coderbunker and has a background in marketing and project management, currently, she focuses on data engineering learning and deep learning.

ABOUT CO-LEARNING

Co-Learning is cooperative learning (co-learning) sessions in a work environment where participants are following advanced facilitators, self-paced online curriculum and helping each other succeed. We create a good environment for learning with peers, offer opportunities to apply skills to real projects and coach new developers to use industry standard practices. Check out our colearning scoreboard on freeCodeCamp athttp://fcc.coderbunker.com/.

PROGRAMS

• Learn front and back end development through freeCodeCamp

• Learn data science through DataCamp

• Learn DevOps best practice through AWS Training

• Become a full stack web developer

• Become a data engineer or scientist

• Become a certified AWS expert

• Collaborate on Open Source Project to reach professional proficiency

Follow these co-learning tracks using high quality and self-paced online courses. For those who completed at least 50% of the learning track, we invite you to join Open Source projects in small teams to experience a professional team workflow. More on projects athttp://github.com/coderbunker

ORGANIZER

Coderbunker is an international community that helps talented developers grow into successful freelancers with their own personal brand. We connect freelancers with customers by helping customers find the right resource at the right price at the right time. Through our community branding, we’ve generated hundreds of such opportunities in the last year.

CO-ORGANIZER

Agora Space is an international co-working office located in Xuhui district, Shanghai. We are engineers, makers, traders, designers, and entrepreneurs working as freelance or running startup or business.

LOCATION

Panyu Lu 1199, Building 8, Xuhui, Shanghai