第2章 python中一切皆对象
本章节首先对比静态语言以及动态语言，然后介绍 python 中最底层也是面向对象最重要的几个概念-object、type和class之间的关系，以此来引出在python如何做到一切皆对象、随后列举python中的常见对象。
2-1 python中一切皆对象
# Python 中一切皆是对象
# 函数和类也是对象，属于Python的一等公民
# 一等公民:
# 1.可以赋值给一个变量
# 2.可以添加到集合对象中
# 3.可以作为参数传递给函数
# 4.可以当做函数的返回值
def ask(name = "mojo"):
print(name)
#函数是对象
my_func = ask
my_func()
class Person:
def __init__(self):
print("好饭不怕晚，加油!")
#类是对象
my_class = Person
my_class()
obj_list = []
obj_list.append(ask)
obj_list.append(Person)
for item in obj_list:
print(item())
# mojo
# None 默认返回
# 好饭不怕晚，加油!
# <__main__.person object at> 类对象
# 装饰器原理
def decorator_func():
print("dec start")
return ask
my_ask = decorator_func()
my_ask("xilepeng")
# dec start
# xilepeng
2-2 type、object和class之间的关系
>>> a = 1
>>> type(a)
>>> type(int)
>>>
type -> int -> 1
>>> class Student:
... pass
...
>>> stu = Student()
>>> type(stu)
>>> type(Student)
>>>
type-> class -> object
object 是最顶层基类，所有类的最顶层类
结论：类是 type 类生成的一个对象
object是所有类都要继承的顶层的基础类，什么都没写，默认继承object
type也是一个类，同时type也是一个对象
>>> Student.__bases__
(,)
>>>
>>> Student.__bases__
(,)
>>> class MyStudent(Student):
... pass
...
>>> MyStudent.__bases__
(,)
>>>
>>> type.__bases__
(,)
>>>
>>> type(object)
>>>
>>> object.__bases__
()
>>>
2-3 python中的内置类型
>>> a = 1
>>> id(a)
4534318224 # a对象的地址
>>>
>>> a = None
>>> b = None
>>> a == b
True
>>>
Python 中常见内置类型：
1.对象三大特性:1.身份(地址id())、2.类型：、3.值
2.None(全局只有一个)
3.数值类型：int,float,complex(复数),bool
迭代类型
5.序列类型：list,(bytes,bytearry二进制序列)，range,tuple,str,arry
6.映射(dict)
7.集合(set,frozenset不可修改的set)
8.上下文管理类型(with )
9.其他(模块类型，class和实例，函数类型，方法类型，代码类型，object对象，type类型)
2-4 本章小结
第3章 魔法函数
本章将会介绍python语言简介语法背后的基石-魔法函数，会通过例子来演示魔法函数对python的影响、最后整体呈现python中的魔法函数来对python做一个概览。
3-1 什么是魔法函数
class Company(object):
def __init__(self, employee_list):
self.employee = employee_list
def __getitem__(self, item):
return self.employee[item]
company = Company(["mojo", "x", "xilepeng"])
employee = company.employee
for em in employee:
print(em)
#对类对象循环遍历，对象调用getitem魔法方法，直到抛出异常为止，结束循环
for em in company:
print(em)
company1 = company[:1]
for em in company1:
print(em)
3-2 python数据模型对python的影响
class Company(object):
def __init__(self, employee_list):
self.employee = employee_list
def __getitem__(self, item):
return self.employee[item]
def __len__(self):
return len(self.employee)
company = Company(["mojo", "x", "xilepeng"])
employee = company.employee
for em in employee:
print(em)
#对类对象循环遍历，对象调用getitem魔法方法，直到抛出异常为止，结束循环
for em in company:
print(em)
company1 = company[:1]
for em in company1:
print(em)
print(len(company1))
print(len(company))
3-3 python魔法函数一览
class Company(object):
def __init__(self, employee_list):
self.employee = employee_list
# 对对象格式化使用
def __str__(self):
return ",".join(self.employee)
#开发模式下使用
def __repr__(self):
return ",".join(self.employee)
company = Company(["mojo"", "x", "xilepeng"])
print(company)
3-4 len函数的特殊性
3-5 本章小结
第4章 深入类和对象
本章节是python面向对象的进阶知识，通过本章的学习会掌握 python 鸭子类型以及鸭子类型对 python 的影响，随后讲解 python 中的抽象基类的运用、python 的 mro 属性查找算法和 super 函数、类变量和对象变量以及数据封装、本章节会讲解对象的自省机制能让大家对对象内部有更进一步的了解、最后是上下文管理协议...
4-1 鸭子类型和多态
class Cat(object):
def say(self):
print("I am a cat")
class Dog(object):
def say(self):
print("I am a dog")
class Duck(object):
def say(self):
print("I am a duck")
# animal = Cat
# animal().say()
#多态: 只需定义类方法，所有类实现了共同的方法，
animal_list = [Cat, Dog, Duck]
for animal in animal_list:
animal().say()
a = ["好饭不怕晚","良缘不怕迟"]
b = ["默默奉献~"]
name_tuple = ["mojo", "man"]
name_set = set()
name_set.add("xilepeng")
name_set.add("go big or go home")
a.extend(b)
a.extend(name_tuple)
print(a)
4-2 抽象基类(abc模块) - 1
class Company(object):
def __init__(self, employee_list):
self.employee = employee_list
def __len__(self):
return len(self.employee)
company = Company(["mojo", "x", "xilepeng"])
print(len(company))
#检查某个类是否有某个方法
print(hasattr(company,"__len__"))
4-3 抽象基类(abc模块) - 2
# 1.我们在某些情况下希望判定某个对象的类型
from collections.abc import Sized
isinstance(company, Sized)
# 2.我们需要强制某个子类必须实现某些方法
# 实现了一个 web框架，集成 cache(redis, cache, memorycache)
#需要设计一个抽象基类，指定子类必须实现某些方法
#如何去模拟一个抽象基类
class CacheBase():
def get(self, key):
raise NotImplementedError
def set(self, key, value):
raise NotImplementedError
class RedisCache(CacheBase):
pass
#运行时抛出异常
redis_cache = RedisCache()
redis_cache.set("key", "value")
#强制子类实现某些方法，否则调用时初始化报异常
import abc
class CacheBase(metaclass=abc.ABCMeta):
@abc.abstractmethod
def get(self, key):
pass
@abc.abstractmethod
def set(self, key, value):
pass
class RedisCache(CacheBase):
pass
redis_cache = RedisCache()
#初始化报异常
4-4 isinstance和type的区别
class A(object):
pass
class B(A):
pass
b = B()
print(isinstance(b, B))
print(isinstance(b, A))
print(type(b) is B ) # is 判断对象id是否相等
print(type(b) is A ) # is 判断对象id是否相等
print(type(b) == B ) # == 判断值是否相等
print(isinstance(b, A))
4-5 类变量和实例变量
class A(object):
aa = 1
def __init__(self, x, y):
self.x = x
self.y = y
a = A(2, 3)
A.aa = 11 #修改类变量
a.aa = 100 # 对象新建aa变量, 优先从下往上查找
print(a.x, a.y, a.aa)
print(A.aa) # 类变量
b = A(3, 5)
print(b.aa)
4-6 类和实例属性的查找顺序—mro查找
# class A:
# name = "A"
# def __init__(self):
# self.name = "obj"
# a = A()
# print(a.name)
# Python3 新式类
# 菱形继承关系
class D:
pass
class C(D):
pass
class B(D):
pass
class A(B,C):
pass
#从底向上继承
print(A.__mro__)
# (, , , , )
class D:
pass
class E:
pass
class B(D):
pass
class C(E):
pass
class A(B,C):
pass
#从底向上继承
print(A.__mro__)
# (, , , , , )
4-7 类方法、静态方法和实例方法
class Date:
#构造函数
def __init__(self, year, month, day):
self.year = year
self.month = month
self.day = day
def toworrow(self):
self.day += 1
#静态方法
@staticmethod
def parse_from_string(date_str):
year,month,day = tuple(date_str.split("-"))
return Date(int(year), int(month), int(day))
#类方法
@classmethod
def from_string(cls, date_str):
year,month,day = tuple(date_str.split("-"))
return cls(int(year), int(month), int(day))
def __str__(self):
return "{year}/{month}/{day}".format(year= self.year, month = self.month, day = self.day)
if __name__ == "__main__":
newday = Date(2020,3,18)
newday.toworrow()
print(newday)
date_str ="2020-3-18"
year,month,day = tuple(date_str.split("-"))
# print(year, month, day)
newday = Date(int(year), int(month), int(day))
print(newday)
# 用staticmethod完成初始化
newday = Date.parse_from_string(date_str)
print(newday)
# 用classmethod完成初始化
newday = Date.from_string(date_str)
print(newday)
4-8 数据封装和私有属性
from class_method import Date
class User:
def __init__(self, birthday):
#私有属性封装，不能通过类对象访问，只能通过类公共方法访问
self.__birthday = birthday
def get_age(self):
return 2018 - self.__birthday.year
if __name__ == "__main__":
user = User(Date(1995,1,3))
print(user._User__birthday)#语言结构化方法
print(user.get_age())
4-9 python对象的自省机制
#自省是通过一定的机制查询到对象的内部结构
from class_method import Date
class Person:
name = "user"
class Student(Person):
def __init__(self, school_name):
self.school_name = school_name
if __name__ == "__main__":
user = Student("mooc")
# 通过__dict__查询属性
print(user.__dict__)
user.__dict__["school_addr"] = "北京市"
print(user.school_addr)
print(Person.__dict__)
print(user.name)
#查看对象所有属性
print(dir(user))
4-10 super真的是调用父类吗？
class A:
def __init__(self):
print("A")
class B(A):
def __init__(self):
print("B")
super().__init__()
class C(A):
def __init__(self):
print("C")
super().__init__()
class D(B, C):
def __init__(self):
print("D")
super().__init__()
#既然我们已经重写了B的构造函数，为什么还要调super?
# 好处：重用代码
#super 执行顺序到底是什么样的？
if __name__ == "__main__":
d = D()
print(D.__mro__)
# D
# B
# C
# A
# (, , , , )
4-11 mixin继承案例-django rest framework
mixin模式特点
1.mixin类功能单一
2.不和基类关联，可以和任意基类组合
3.在mixin中不要使用super(根据mro循环调用)
4-12 python中的with语句
# try:
# print("code started")
# raise KeyError
# except KeyError as e:
# print("key error")
def exe_try():
try:
print("code started")
raise KeyError
return 1 # 压入栈
except KeyError as e:
print("key error")
return 2
else:
print("other error")
return 3
finally: # 资源释放
print("finally")
return 4 # 如果finally有 return ,返回，否则调用之前语句
# 出栈
if __name__ == "__main__":
result = exe_try()
print(result) # 4
# 上下文管理器协议
class Sample:
def __enter__(self):
print("enter:获取资源")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print("exit：释放资源")
def do_something(self):
print("doing something")
with Sample() as sample:
sample.do_something()
# enter:获取资源
# doing something
# exit：释放资源
4-13 contextlib简化上下文管理器
import contextlib
# 将函数(必须是生成器 yield)修饰成上下文管理器
@contextlib.contextmanager
def file_open(file_name):
print("file open")
yield {}
print("file end")
with file_open("mojo") as file_opened:
print("file processing")
# file open
# file processing
# file end
4-14 本章小结
第5章 自定义序列类
本章节在讲解 python 的序列协议后进一步讲解 python 中序列的类型以及序列协议中的魔法函数，之后实现了自己的可以切片的序列、在本章中我们也会接触 bisect 和列表推导式、生成器表达式和字典推导式等，经过本章的学习之后大家会知道如何去定义可以像list一样使用方便的类以及明白 django 中的 queryset 的核心...
5-1 python中的序列分类
5-2 python中序列类型的abc继承关系
5-3 list中extend方法区别
mylist = []
mylist.append(1)
mylist.append("a")
print(mylist)
a = [1, 2]
c = a+ [3, 4] #只有list类型
print(c)
# 就地加,任意序列类型
# a += [3, 4]
# print(a)
a += (3, 4)
a.extend(range(3)) #迭代添加
a.append([1,2]) #将数组变为一个值添加，而不是迭代
a.append((1, 2))
print(a)
# [1, 'a']
# [1, 2, 3, 4]
# [1, 2, 3, 4, 0, 1, 2, [1, 2], (1, 2)]
5-4 实现可切片的对象
未调试通过
import numbers
class Group:
#支持切片操作
def __init__(self, group_name, company_name, staffs):
self.group_name = group_name
self.company_name = company_name
self.staffs = staffs #变成数组
def __reverse__(self):
self.staffs.reverse()
def __getitem__(self, item):
cls = type(self)
if isinstance(item, slice):
return cls(group_name = self.group_name, company_name = self.company_name, staffs = self.staffs[item])
elif isinstance(item, numbers.Integral):
return cls(group_name = self.group_name, company_name = self.company_name, [staffs = self.staffs[item]])
def __len__(self):
return len(self.staffs)
def __iter__(self):
return iter(self.staffs)
def __contains__(self,item):
if item in self.staffs:
return True
else:
return False
staffs = ["bobby1", "imooc", "bobby2"]
group = Group(company_name = "imooc", group_name = "user", staffs = staffs)
sub_group = group[:2]
print(sub_group)
5-5 bisect维护已排序序列
import bisect
# 用来处理已排序的序列，用来维持已排序的序列，升序
#二分查找
insert_list = []
bisect.insort(insert_list, 3)
bisect.insort(insert_list, 2)
bisect.insort(insert_list, 1)
bisect.insort(insert_list, 4)
print(bisect.bisect_left(insert_list, 3)) # 查找插入位置
print(insert_list)
5-6 什么时候我们不该使用列表
import array
# array list 区别：array 只能存放指定的数据类型
# array 性能比list高很多
my_array = array.array("i")
my_array.append(1)
my_array.append('asf')
5-7 列表推导式、生成器表达式、字典推导式
# 列表生成式(列表推导式)
# 1.提取1-20之间的奇数
odd_list = []
for i in range(21):
if i % 2 == 1:
odd_list.append(i)
odd_list = [i for i in range(21) if i % 2 == 1]
# 2.逻辑复杂的情况
def hadle_item(item):
return item * item
odd_list = [hadle_item(i) for i in range(21) if i % 2 == 1]
# 3.列表生成式性能高于列表
# print(type(odd_list))
# print(odd_list)
# 生成式表达式
odd_gen = (i for i in range(21) if i % 2 == 1)
print(type(odd_gen))
print(odd_gen)
# for item in odd_gen:
# print(item)
odd_list = list(odd_gen)
print(type(odd_list))
print(odd_list)
# 字典推导式
my_dict = {"mojo1":18, "x":19}
reversed_dict = {value:key for key,value in my_dict.items()}
print(reversed_dict)
# 集合推导式
# my_set = set(my_dict.keys())
my_set = {key for key, value in my_dict.items()}
print(type(my_set))
print(my_set)
5-8 本章小结
第6章 深入python的set和dict
因为 dict 的高性能，dict 在 python 内部被大量应用。本章节我们会首先通过例子演示dict 和 list 之间的性能差异，以及讲解 dict 高性能背后的原理，我们也将接触到散列表以及可散列类型，最后我们我们会知道 set 和 frozenset 的区别。...
6-1 dict的abc继承关系
from collections import Mapping, MutableMapping
# dict 属于mapping 类型
a = {}
print(isinstance(a, MutableMapping))
6-2 dict的常用方法
d = {
"mojo":{"company":{"x":"x1"}}
}
# d.clear()
# print(d)
# copy 返回浅拷贝,同时改变了d
# new_dict = d.copy()
# new_dict["mojo"]["company"] = "x522"
# pass
# 深copy，不改变d
import copy
new_dict = copy.deepcopy(d)
new_dict["mojo"]["company"] = "x522"
new_list = ["bobby1", "bobby2"]
new_dict = dict.fromkeys(new_list, {"company":"mooc"})
value = new_dict.get("bobby1", {})
# for key, value in new_dict.items():
# print(key,value)
default_value = new_dict.setdefault("alien","imooc")
new_dict.update({"xilepeng":"x"})
new_dict.update(x2= "xi", x3 = "xilepeng")
new_dict.update([("bobby3", "mooc")])
pass
6-3 dict的子类
# 不建议继承list dict
class Mydict(dict): # c语言写的dict 不会调用覆盖方法
def __setitem__(self, key, value):
super().__setitem__(key, value*2)
my_dict = Mydict(one = 1) # 并没有调用魔法函数
# my_dict["one"] = 1
print(my_dict)
from collections import UserDict
class Mydict(UserDict): # 会调用覆盖方法
def __setitem__(self, key, value):
super().__setitem__(key, value*2)
my_dict = Mydict(one = 1) #调用魔法函数
# my_dict["one"] = 1
print(my_dict)
from collections import defaultdict
my_dict = defaultdict(dict)
my_value = my_dict["mojo"]
pass
6-4 set和frozenset
# set 集合 fronzenset不可变集合 无序，不重复
# s = set("abcdee")
# s = set(['a', 'b', 'c'])
s = {'a', 'b'}
s.add('c')
print(type(s))
print(s)
# s = frozenset("abcde") #定义好后，值不可变，可以作为 dict 的 key
# print(s)
another_set = set("cdef")
# s.update(another_set)
# print(s)
res_set = s.difference(another_set)
res_set = s - another_set
res_set = s & another_set
res_set = s | another_set
print(res_set)
if "c" in res_set:
print("c in res_set")
print(s.issubset(res_set))
6-5 dict和set的实现原理
from random import randint
def load_list_data(total_nums, target_nums):
"""
从文件中读取数据，以list的方式返回
:param total_nums: 读取的数量
:param target_nums: 需要查询的数据的数量
"""
all_data = []
target_data = []
file_name = "G:/慕课网课程/AdvancePython/fbobject_idnew.txt"
with open(file_name, encoding="utf8", mode="r") as f_open:
for count, line in enumerate(f_open):
if count < total_nums:
all_data.append(line)
else:
break
for x in range(target_nums):
random_index = randint(0, total_nums)
if all_data[random_index] not in target_data:
target_data.append(all_data[random_index])
if len(target_data) == target_nums:
break
return all_data, target_data
def load_dict_data(total_nums, target_nums):
"""
从文件中读取数据，以dict的方式返回
:param total_nums: 读取的数量
:param target_nums: 需要查询的数据的数量
"""
all_data = {}
target_data = []
file_name = "G:/慕课网课程/AdvancePython/fbobject_idnew.txt"
with open(file_name, encoding="utf8", mode="r") as f_open:
for count, line in enumerate(f_open):
if count < total_nums:
all_data[line] = 0
else:
break
all_data_list = list(all_data)
for x in range(target_nums):
random_index = randint(0, total_nums-1)
if all_data_list[random_index] not in target_data:
target_data.append(all_data_list[random_index])
if len(target_data) == target_nums:
break
return all_data, target_data
def find_test(all_data, target_data):
#测试运行时间
test_times = 100
total_times = 0
import time
for i in range(test_times):
find = 0
start_time = time.time()
for data in target_data:
if data in all_data:
find += 1
last_time = time.time() - start_time
total_times += last_time
return total_times/test_times
if __name__ == "__main__":
all_data, target_data = load_list_data(10000, 1000)
# all_data, target_data = load_list_data(100000, 1000)
# all_data, target_data = load_list_data(1000000, 1000)
# all_data, target_data = load_dict_data(10000, 1000)
# all_data, target_data = load_dict_data(100000, 1000)
# all_data, target_data = load_dict_data(1000000, 1000)
last_time = find_test(all_data, target_data)
print(last_time)
# 在list 中查找元素随着list增大而增大
# 在dict 中查找元素不会随着dict 增大而增大
6-6 本章小结
第7章 对象引用、可变性和垃圾回收
本章节是偏理论的章节，却是我们进一步理解 pytho n以及排查各种隐含的 bug 最重要的章节，本章的对象引用、可变性和垃圾回收会让我们对 python 的变量本质有更进一步的加深，本章节会让我们在编码的过程中尽量避免各种坑以及出错后有排错的经验。...
7-1 python中的变量是什么
# Python变量实质上是一个指针,便利贴
a = 1 #a贴在1上面
a = "abc" # 先生成对象，然后生成便利贴
a = [1,2,3]
b = a
print(a is b) # a b 为同一个对象
print(id(a),id(b)) # a对象所指向的 id
# True
b.append(4)
print(a)
7-2 ==和is的区别
a = [5, 2, 2]
b = [5, 2, 2] #重新声明一个对象
print(id(a),id(b)) # a对象所指向的 id
# 4335517824 4335519936
print(a is b) # 是不是同一对象判断
# False
print(a == b) # True 值的判断
# __eq__,不同对象，但值相等
# python 内部小整数优化机制
a = 1
b = 1
print(id(a),id(b)) # a对象所指向的 id
# 4562897168 4562897168
print(a is b)
# True
class People:
pass
person = People()
if type(person) is People: # People 全局唯一的对象
print("yes")
7-3 del语句和垃圾回收
# Python 垃圾回收机制：采用引用计数算法
a = object()
b = a
del a # a 的计数器减1
print(b)
#
print(a)
# NameError: name 'a' is not defined
7-4 一个经典的参数错误
def add(a, b):
a += b # 就地修改a ,a传进来a的引用, list 是可变对象
return a
if __name__ == "__main__":
a = 1
b = 2
c = add(a,b)
print(c)
print(a, b)
a = [1, 2]
b = [3, 4]
c = add(a, b)
print(c)
print(a, b)
a = (1, 2)
b = (3, 4)
c = add(a, b)
print(c)
print(a, b)
class Company:
def __init__(self, name, staffs= []):
self.name = name
self.staffs = staffs
def add(self, staff_name):
self.staffs.append(staff_name)
def remove(self, staff_name):
self.staffs.remove(staff_name)
if __name__ == "__main__":
com1 = Company("bytedance",["mojo1", "mojo2"])
com1.add("mojo3")
com1.remove("mojo1")
print(com1.staffs) # ['mojo2', 'mojo3']
com2 = Company("com2")
com2.add("mojo2")
print(com2.staffs) # ['mojo2']
print(Company.__init__.__defaults__)
# (['mojo2'],) 默认指向对象
com3 = Company("mojo3")
com3.add("mojo3")
print(com3.staffs) # ['mojo2', 'mojo3']
print(com2.staffs is com3.staffs) # 是同一对象
# True
# 原因： com2 com3都没有传递 staffs (list 可变对象)，默认使用同一[]可变对象
7-5 本章小结
第8章 元类编程
元类在 python 高级工程师面试中会被经常问到、元类作为 python 中一个高级特性，熟练使用元类能不仅让我们写出更加优雅和可控性更好的代码还能进一步加深我们对python 的理解、本章节我们将会理解更多的 python 面向对象的高级特性比如property 以及属性描述符、getattr和getattribute等等，这些让我们可...
8-1 property动态属性
from datetime import date,datetime
class User:
def __init__(self, name, birthday):
self.name = name
self.birthday = birthday
self._age = 0
# def get_age(self):
# return datetime.now().year - self.birthday.year
@property
def age(self):
return datetime.now().year - self.birthday.year
@age.setter
def age(self, value):
self._age = value
if __name__ == "__main__":
user = User("mojo", date(year = 1995, month = 3, day = 26))
# print("in {} file".format(__file__))
# print(user.get_age())
user.age = 30
print(user.age)
print(user._age)
8-2 getattr、getattribute魔法函数
# __getattr__ 在查找不到属性时调用
# __getattribute__
from datetime import date,datetime
class User:
def __init__(self, info = {}):
self.info = info
# 在查找不到属性时调用
def __getattr__(self, item):
return self.info[item]
# def __getattribute__(self, item):
# return "所有属性的入口"
if __name__ == "__main__":
user = User(info = {"company_name":"imooc", "name":"mojo"})
# print(user.age)
# AttributeError: 'User' object has no attribute 'age'
print(user.company_name)
print(user.name)
8-3 属性描述符和属性查找过程
import numbers
class IntField:
# 属性描述符
def __get__(self, instance, owner):
return self.value
def __set__(self, instance, value):
if not isinstance(value, numbers.Integral):
raise ValueError("int value need")
if value < 0:
raise ValueError("positive value need")
self.value = value
def __delete__(self, instance):
pass
class NonDataIntField:
# 非数据描述符
def __get__(self, instance, owner):
return self.value
class User:
age = IntField() # 属性描述符对象 {}
# age = NonDataIntField() # {'age': 30}
if __name__ == "__main__":
user = User()
# user.age = 30
user.__dict__['age'] = "abc" # {'age': 'abc'}
print(user.age)
print(user.__dict__)
# user.age = -1
# user.age = "a"
# print(user.age)
# print(getattr(user, 'age')) # print(user.age) #等价
8-4 new和init的区别
class User:
def __new__(cls, *args, **kwargs):
print(" in new")
return super().__new__(cls)
def __init__(self, name):
print("in init")
# new 是用来控制对象的生成过程，在对象生成之前
# init 是用来完善对象的，
# 如果new方法不返回对象，则不会调用init函数
if __name__ == '__main__':
usr = User(name='mojo')
8-5 自定义元类
# class 也是对象， type 是创建类的类
def create_class(name):
if name == "user":
class User:
def __str__(self):
return "user"
return User
elif name == "company":
class Company:
def __str__(self):
return "company"
return Company
# type 动态创建类
# User = type("User", (), {})
def say(self):
return "I am user"
class BaseClass:
def answer(self):
return "I am BaseClass"
# 什么是元类？元类是创建类的类 ， 对象
class MetaClass(type):
def __new__(cls, *args, **kwargs):
return super().__new__(cls, *args, **kwargs)
class User(metaclass = MetaClass):
def __init__(self, name):
self.name = name
def __str__(self):
return "user"
# python中类是实例化过程，会首先寻址metaclass, 通过metaclass去创建user类
# type创建类对象
if __name__ == "__main__":
# MyClass = create_class("user")
# my_obj = MyClass()
# print(my_obj)
# print(type(my_obj))
# User = type("User", (BaseClass,), {"name":"user", "say":say})
my_obj = User(name= "mojo")
# print(my_obj.say())
print(my_obj)
8-6 通过元类实现orm-1
8-7 通过元类实现orm-2
8-8 本章小结
第9章 迭代器和生成器
深刻理解生成器是理解协程的基础、迭代器和生成器作为 python 难以理解的功能，很多人对其区别以及使用都是模棱两可，本章节我们会全面理解迭代器协议以及生成器和迭代器之间的关系，我们会重点讲解生成器的原理，让我们更清楚我们在什么时候应该使用生成器，本章节我会通过几个例子加深大家对生成器的理解和使用。...
9-1 python中的迭代协议
# 什么是迭代协议？
# 迭代器是什么？
# 迭代器是访问集合类元素的一种方式，一般用来遍历数据
# 迭代器和下标的访问方式不一样，迭代器是不能返回的，迭代器提供了一种惰性数据访问方式
# [] list __iter__
from collections.abc import Iterable,Iterator
alist = [1, 2]
print(isinstance(alist, Iterable))
print(isinstance(alist, Iterator))
# True
# False
9-2 什么是迭代器和可迭代对象
# 什么是迭代协议？
# 迭代器是什么？
# 迭代器是访问集合类元素的一种方式，一般用来遍历数据
# 迭代器和下标的访问方式不一样，迭代器是不能返回的，迭代器提供了一种惰性数据访问方式
# [] list __iter__
from collections.abc import Iterable,Iterator
alist = [1, 2]
print(isinstance(alist, Iterable))
print(isinstance(alist, Iterator))
# True
# False
iter_ator = iter(alist) # iter()方法返回迭代器
print(isinstance(iter_ator, Iterator))
# True
class Company:
def __init__(self, employee_list):
self.employee = employee_list
def __iter__(self):
return MyIterator(self.employee)
# def __getitem__(self, item):
# return self.employee[item]
class MyIterator(Iterator):
def __init__(self, employee_list):
self.iter_list = employee_list
self.index = 0
def __next__(self):# 真正返回迭代值的逻辑
try:
word = self.iter_list[self.index]
except IndexError:
raise StopIteration
self.index += 1
return word
if __name__ == "__main__":
company = Company(["mojo1", "mojo2"])
my_itor = iter(company)
# while True:
# try:
# print(next(my_itor))
# except StopIteration:
# pass
for item in company:
print(item)
9-3 生成器函数的使用
9-4 python是如何实现生成器的
# python中函数的工作原理
import inspect
frame = None
def foo():
bar()
def bar():
global frame
frame = inspect.currentframe()
# python.exe 会用一个叫做PyEval_EvalFramEx( c 函数)去执行foo函数
# 首先会创建一个栈帧(stack fream)
"""
python一切皆对象，栈帧对象，Python运行前编译成字节码，编译成字节码对象,在栈帧对象的上下文运行字节码，字节码全局唯一
当foo 调用子函数bar,又会创建一个栈帧
所有栈帧都分配在堆内存上，这就决定了栈帧可以独立于调用者存在
"""
# import dis
# # print(dis.dis(foo))
foo()
print(frame.f_code.co_name)
caller_frame = frame.f_back
print(caller_frame.f_code.co_name)
def gen_func():
yield 1
name = "mojo"
yield 2
age = "25"
return "mooc"
gen = gen_func()
import dis
print(dis.dis(gen))
print(gen.gi_frame.f_lasti)
print(gen.gi_frame.f_locals)
next(gen)
print(gen.gi_frame.f_lasti)
print(gen.gi_frame.f_locals)
next(gen)
print(gen.gi_frame.f_lasti)
print(gen.gi_frame.f_locals)
next(gen)
print(gen.gi_frame.f_lasti)
print(gen.gi_frame.f_locals)
9-5 生成器在UserList中的应用
9-6 生成器如何读取大文件
# 500 G,特殊：一行
# f = open()
# for line in f:
# f.readlines()
def myreadlines(f, newline):
buf = ""
while True:
while newline in buf:
pos = buf.index(newline)
yield buf[:pos]
buf = buf[pos + len(newline):]
chunk = f.read(4096)
if not chunk:
#说明已经读到文件末尾
yield buf
break
buf += chunk
with open("input.txt") as f:
for line in myreadlines(f, "{|}"):
print(line)
sdf sdfaslkfjskaljflksf {|} ,lsdjflskjdfl;sakfjlks{|},saldkfjlaskdfj;lsdkjf;sadjf;{|}skldjflsdkfjsdlkjflk
9-7 本章小结
第10章 python socket编程
本章节我会从 http、socket、tcp 协议开始讲起，通过 socket 方式实现客户端和服务端让大家名明白聊天类软件的核心、要想深刻理解 web 编程、我们必须知道 socket 编程，本章节我们将通过多线程+ socket 的方式实现支持并发的服务端、最后通过 socket 模拟 http 的请求来实现为后续的异步 IO 打下并发的基...
10-1 弄懂 HTTP、Socket、TCP 这几个概念
10-2 socket 和 server 实现通信
socket_server.py
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # ipv4网络， udp
server.bind(('0.0.0.0', 8000))
server.listen()
sock, addr = server.accept()
# 获取从客户端发送的数据
# 一次获取1k数据
data = sock.recv(1024)
print(data.decode("utf8"))
sock.send("hello {}".format(data.decode("utf8")).encode("utf8"))
server.close()
sock.close()
socket_client.py
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # ipv4网络， udp
client.connect(('127.0.0.1', 8000))
client.send("x".encode("utf8"))
data = client.recv(1024)
print(data.decode("utf8"))
client.close()
10-3 socket 实现聊天和多用户连接
单用户聊天
socket_server.py
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # ipv4网络， udp
server.bind(('0.0.0.0', 8000))
server.listen()
sock, addr = server.accept()
# 获取从客户端发送的数据
# 一次获取1k数据
while True:
data = sock.recv(1024)
print(data.decode("utf8"))
re_data = input()
sock.send(re_data.encode("utf8"))
socket_client.py
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # ipv4网络， udp
client.connect(('127.0.0.1', 8000))
while True:
re_data = input()
client.send(re_data.encode("utf8"))
data = client.recv(1024)
print(data.decode("utf8"))
多用户连接
socket_server.py
socket_client.py
10-4 socket 模拟 http请求
# requests -> urlib -> socket
import socket
from urllib.parse import urlparse
def get_url(url):
# 通过socket 请求html
url = urlparse(url)
host = url.netloc
path = url.path
if path == "":
path = "/"
# 建立socket连接
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('host', 80))
client.send("GET {}HTTP/1.1\r\nHost:{}\r\nConnection:close\r\n\r\n".format(path, host).encode("utf8"))
data = b""
while True:
d = client.recv(1024)
if d:
data += d
else:
break
data = data.decode("utf-8")
html_data = data.split("\r\n\r\n")[1]
print(html_data)
client.close()
if __name__ == "__main__":
get_url("https://www.baidu.com")
10-5 本章小结
第11章 多线程、多进程和线程池编程
多线程、多进程编程一直是面试中被问到的高频问题，本章节我们将从 GIL 开始讲解多线程以及多进程的应用场景、之后详细的介绍多线程的编码、线程间通信以及线程的同步- Lock\Rlock\Condition，通过对 condition 的源码分析加深大家对条件变量的理解，接着通过线程池 ThreadPoolExecutor 的使用和源码分析加深大家对...
11-1 python 中的 GIL
# gil : global interpreter lock 全局解释器(cpython)
# python 中一个线程对应c语言中的一个线程
# gil 使得同一时刻只有一个线程在一个cpu上执行字节码,无法将多个线程映射到多个cpu上执行
# import dis
# def add(a):
# a += 1
# return a
# print(dis.dis(add))
# gil 会根据执行的字节码行数以及时间片释放gil,gil在遇到io操作的时候主动释放
import threading
total = 0
def add():
global total
for i in range(1000000):
total += 1
def desc():
global total
for i in range(1000000):
total -= 1
thread1 = threading.Thread(target = add)
thread2 = threading.Thread(target = desc)
thread1.start()
thread2.start()
thread1.join()
thread2.join()
print(total)
# ============ RESTART: /Users/mojo/Documents/Python/chapter11/GIL.py ============
# -266220
# >>>
# ============ RESTART: /Users/mojo/Documents/Python/chapter11/GIL.py ============
# 44580
# >>>
# ============ RESTART: /Users/mojo/Documents/Python/chapter11/GIL.py ============
# 498497
# >>>
11-2 多线程编程 - threading
# 对于io操作来说，多线程和多进程性能差别不大
# 线程调度方式：
# 1.通过Thread类实例化
import threading
import time
def get_detail_html(url):
print('get detail html started\n')
time.sleep(2)
print('get detail html end\n')
def get_detail_url(url):
print('get detail url started\n')
time.sleep(4)
print('get detail url end\n')
if __name__ == "__main__":
thread1 = threading.Thread(target=get_detail_html, args=("",))
thread2 = threading.Thread(target=get_detail_url, args=("",))
start_time = time.time()
#thread1.setDaemon(True) #将线程1设置为守护线程，当主线程退出后，将守护线程也关闭掉
thread2.setDaemon(True)
thread1.start()
thread2.start()
# thread1.join() # 等线程执行完后执行后面代码
# thread2.join()
# 当主线程退出时，子线程kill掉
print("last time:{}".format(time.time()-start_time))
import threading
import time
class GetDetialHtml(threading.Thread):
def __init__(self, name):
super().__init__(name=name)
def run(self):
print('get detail html started\n')
time.sleep(2)
print('get detail html end\n')
class GetDetailUrl(threading.Thread):
def __init__(self, name):
super().__init__(name=name)
def run(self):
print('get detail url started\n')
time.sleep(4)
print('get detail url end\n')
if __name__ == "__main__":
thread1 = GetDetialHtml("get_detail_html")
thread2 = GetDetailUrl("get_detail_url")
start_time = time.time()
#thread1.setDaemon(True) #将线程1设置为守护线程，当主线程退出后，将守护线程也关闭掉
# thread2.setDaemon(True)
thread1.start()
thread2.start()
# thread1.join() # 等线程执行完后执行后面代码
# thread2.join()
# 当主线程退出时，子线程kill掉
print("last time:{}".format(time.time()-start_time))
11-3 线程间通信 - 共享变量和 Queue
# # 对于io操作来说，多线程和多进程性能差别不大
# # 线程调度方式：
# # 1.通过Thread类实例化
import threading
import time
from chapter11 import variable
def get_detail_html(detial_url_list):
#爬取文章详情页
detial_url_list = variable.detial_url_list
while True:
if len(variable.detial_url_list):
url = variable.detial_url_list.pop()
print('get detail html started\n')
time.sleep(2)
print('get detail html end\n')
def get_detail_url(variable.detial_url_list):
# 爬取文章列表页
while True:
print('get detail url started\n')
time.sleep(4)
print('get detail url end\n')
# 1.线程通信方式-共享变量
if __name__ == "__main__":
get_detail_url = threading.Thread(target=get_detail_url, args=(get_detail_url,))
for i in range(10):
html_thread = threading.Thread(target=get_detail_html, args=(get_detail_html,))
html_thread.start()
start_time = time.time()
#thread1.setDaemon(True) #将线程1设置为守护线程，当主线程退出后，将守护线程也关闭掉
# thread2.setDaemon(True)
# thread1.start()
# thread2.start()
# thread1.join() # 等线程执行完后执行后面代码
# thread2.join()
# 当主线程退出时，子线程kill掉
# print("last time:{}".format(time.time()-start_time))
# 2.通过继承 Thread 来实现多线程
# import threading
# import time
# class GetDetialHtml(threading.Thread):
# def __init__(self, name):
# super().__init__(name=name)
# def run(self):
# print('get detail html started\n')
# time.sleep(2)
# print('get detail html end\n')
# class GetDetailUrl(threading.Thread):
# def __init__(self, name):
# super().__init__(name=name)
# def run(self):
# print('get detail url started\n')
# time.sleep(4)
# print('get detail url end\n')
# if __name__ == "__main__":
# thread1 = GetDetialHtml("get_detail_html")
# thread2 = GetDetailUrl("get_detail_url")
# start_time = time.time()
# #thread1.setDaemon(True) #将线程1设置为守护线程，当主线程退出后，将守护线程也关闭掉
# # thread2.setDaemon(True)
# thread1.start()
# thread2.start()
# # thread1.join() # 等线程执行完后执行后面代码
# # thread2.join()
# # 当主线程退出时，子线程kill掉
# print("last time:{}".format(time.time()-start_time))
# 通过queue 方式进行线程间同步
from queue import Queue
import threading
import time
def get_detail_html(queue):
#爬取文章详情页
while True:
url = queue.get()
print('get detail html started\n')
time.sleep(2)
print('get detail html end\n')
def get_detail_url(queue):
# 爬取文章列表页
while True:
print('get detail url started\n')
time.sleep(4)
for i in range(20):
queue.put("http://projectsedu.com/{id}".format(id = i))
print('get detail url end\n')
# 1.线程通信方式-共享变量
if __name__ == "__main__":
detail_url_queue = Queue(maxsize = 1000)
get_detail_url = threading.Thread(target=get_detail_url, args=(detail_url_queue,))
for i in range(10):
html_thread = threading.Thread(target=get_detail_html, args=(detail_url_queue,))
html_thread.start()
start_time = time.time()
detail_url_queue.task_done()
detail_url_queue.join()
# 当主线程退出时，子线程kill掉
print("last time:{}".format(time.time()-start_time))
11-4 线程同步 - Lock、RLock
11-5 线程同步 - condition 使用以及源码分析
import threading
# 条件变量，用于复杂的线程间同步
class XiaoAi(threading.Thread):
def __init__(self, lock):
super().__init__(name="小爱")
self.lock = lock
def run(self):
self.lock.acquire()
print("{}:在".format(self.name))
self.lock.release()
self.lock.acquire()
print("{}:好啊".format(self.name))
self.lock.release()
class TianMao(threading.Thread):
def __init__(self,lock):
super().__init__(name="天猫精灵")
self.lock = lock
def run(self):
self.lock.acquire()
print("{}:小爱同学".format(self.name) )
self.lock.release()
self.lock.acquire()
print("{}:我们来对古诗吧".format(self.name) )
self.lock.release()
if __name__ == "__main__":
lock = threading.Lock()
xiaoai = XiaoAi(lock)
tianmao = TianMao(lock)
tianmao.start()
xiaoai.start()
#>>> 天猫精灵:小爱同学
#天猫精灵:我们来对古诗吧
#小爱:在
#小爱:好啊
# import threading
# # 条件变量，用于复杂的线程间同步
# class XiaoAi(threading.Thread):
# def __init__(self, lock):
# super().__init__(name="小爱")
# self.lock = lock
# def run(self):
# self.lock.acquire()
# print("{}:在".format(self.name))
# self.lock.release()
# self.lock.acquire()
# print("{}:好啊".format(self.name))
# self.lock.release()
# class TianMao(threading.Thread):
# def __init__(self,lock):
# super().__init__(name="天猫精灵")
# self.lock = lock
# def run(self):
# self.lock.acquire()
# print("{}:小爱同学".format(self.name) )
# self.lock.release()
# self.lock.acquire()
# print("{}:我们来对古诗吧".format(self.name) )
# self.lock.release()
# if __name__ == "__main__":
# lock = threading.Lock()
# xiaoai = XiaoAi(lock)
# tianmao = TianMao(lock)
# tianmao.start()
# xiaoai.start()
# #>>> 天猫精灵:小爱同学
# #天猫精灵:我们来对古诗吧
# #小爱:在
# #小爱:好啊
#通过condition协同读诗
import threading
from threading import Condition
# 条件变量，用于复杂的线程间同步
class XiaoAi(threading.Thread):
def __init__(self, cond):
super().__init__(name="小爱")
self.cond = cond
def run(self):
with self.cond:
self.cond.wait()
print("{}:在".format(self.name))
self.cond.notify()
self.cond.wait()
print("{}:好啊".format(self.name))
self.cond.notify()
self.cond.wait()
print("{}:君住长江尾".format(self.name))
self.cond.notify()
class TianMao(threading.Thread):
def __init__(self,cond):
super().__init__(name="天猫精灵")
self.cond = cond
def run(self):
"""
with self.cond:
print("{}:小爱同学".format(self.name) )
self.cond.notify() #通知
self.cond.wait()
print("{}:我们来对古诗吧".format(self.name) )
self.cond.notify() #通知
self.cond.wait()
print("{}:我在长江头".format(self.name) )
self.cond.notify() #通知
self.cond.wait()
"""
self.cond.acquire()
print("{}:小爱同学".format(self.name) )
self.cond.notify() #通知
self.cond.wait()
print("{}:我们来对古诗吧".format(self.name) )
self.cond.notify() #通知
self.cond.wait()
print("{}:我在长江头".format(self.name) )
self.cond.notify() #通知
self.cond.wait()
self.cond.release()
if __name__ == "__main__":
cond = threading.Condition()
xiaoai = XiaoAi(cond)
tianmao = TianMao(cond)
#启动顺序很重要
#在调用 with cond:后才能调用 wait notify 方法
#condition有2层锁，一把底层锁会在线程调用了wait 方法的时候释放，
#上面的锁会在每次调用wait的时候分配一把并放入到cond的等待队列中，等到notify方法唤醒
xiaoai.start()
tianmao.start()
11-6 线程同步 - Semaphore 使用以及源码分析
# Semaphore 是用于控制进入数量的锁
# 文件读写：写一般只适用于一个线程写，读可以允许多个
import threading
import time
class HtmlSpider(threading.Thread):
def __init__(self, url,sem):
super().__init__()
self.url = url
self.sem = sem
def run(self):
time.sleep(2)
print("got html text success\n")
self.sem.release()
class UrlProducer(threading.Thread):
def __init__(self, sem):
super().__init__()
self.sem = sem
def run(self):
for i in range(20):
self.sem.acquire()
html_thread = HtmlSpider("https://baidu.com/{}".format(i), self.sem)
html_thread.start()
if __name__ == "__main__":
sem = threading.Semaphore(3)
url_producer = UrlProducer(sem)
url_producer.start()
11-7 ThreadPoolExecutor线程池
# futures 进行多线程 多进程编程
# 线程池，为什么要用线程池？
# 1.主线程中可以获取某一个线程的状态或某一个任务的状态，以及返回值
# 2.当一个线程完成时主线程立即知道
# 3.futures 可以让多线程多进程编码接口一致
import time
from concurrent.futures import ThreadPoolExecutor, as_completed, wait
def get_html(times):
time.sleep(times)
print("get page {} success".format(times))
return times
executor = ThreadPoolExecutor(max_workers=2)
# 要获取已经成功的task的返回
urls = [3, 2, 4]
all_task = [executor.submit(get_html, (url)) for url in urls]
# wait(all_task, return_when = FIRST_COMPLETED) # 等所有线程执行完成再执行wait()
# print("main")
for future in as_completed(all_task):
data = future.result()
print("get {} page".format(data))
# 通过executor 的map获取已经完成的task的值
# for data in executor.map(get_html, urls):
# print("get {} page".format(data))
# 通过 submit函数提交执行的函数到线程池中,submit是立即返回，非阻塞
# task1 = executor.submit(get_html, (3))
# task2 = executor.submit(get_html, (2))
# done() 用于判断某个任务是否完成
# print(task1.done())
# print(task2.cancel()) # 如果任务未执行，可以取消
# time.sleep(3)
# print(task1.done())
# print(task1.result())
11-8 ThreadPoolExecutor源码分析
11-9 多线程和多进程对比
# 多进程编程
# 耗cpu操作，用多进程编程， 对于io操作来说，使用多线程编程，进程切换代价高于线程
# 1.耗cpu操作，多进程优与多线程
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from concurrent.futures import ProcessPoolExecutor
def fib(n):
if n<=2: return 1
return fib(n-1) + fib(n-2)
if __name__ == '__main__':
# with ThreadPoolExecutor(3) as executor:
with ProcessPoolExecutor(3) as executor:
urls = [3, 2, 4]
all_task = [executor.submit(fib, (num)) for num in range(25,40)]
start_time = time.time()
for future in as_completed(all_task):
data = future.result()
print("exe result {}".format(data))
print("last time is {}".format(time.time() - start_time))
# 2. 对于io操作来说，多线程优与多进程
import time
def random_sleep(n):
time.sleep(n)
return n
if __name__ == '__main__':
# with ThreadPoolExecutor(3) as executor:
with ProcessPoolExecutor(3) as executor:
urls = [3, 2, 4]
all_task = [executor.submit(random_sleep, (num)) for num in [2]*30]
start_time = time.time()
for future in as_completed(all_task):
data = future.result()
print("exe result {}".format(data))
print("last time is {}".format(time.time() - start_time))
11-10 multiprocessing 多进程编程
# import os
# import time
#
# pid = os.fork() #子进程
# print("mojo")
# if pid == 0:
# print('子进程{},父进程{}'.format(os.getpid(), os.getppid() ) )
# else:
# print('我是父进程：{}'.format(pid))
# time.sleep(2)
from concurrent.futures import ProcessPoolExecutor
import multiprocessing
import time
# 多进程编程
def get_html(n):
time.sleep(2)
print("sub_progress success")
return n
if __name__ == '__main__':
# progress = multiprocessing.Process(target=get_html, args=(2,))
# print(progress.pid)
# progress.start()
# progress.join()
# print("main progress end")
# print(progress.pid)
# 使用线程池
pool = multiprocessing.Pool()
res = pool.apply_async(get_html, args=(3,))
# # 等待所有任务完成
# pool.close()
# pool.join()
# print(res.get())
# imap
for result in pool.imap(get_html, [1,5,3]):
print("{} sleep success".format(result))
# for result in pool.imap_unordered(get_html, [1,5,3]):
# print("{} sleep success".format(result))
11-11 进程间通信 - Queue、Pipe，Manager
# from multiprocessing import Process, Queue
# import time
#
# def producer(queue):
# queue.put("a")
# time.sleep(2)
#
# def consumer(queue):
# time.sleep(2)
# data = queue.get()
# print(data)
#
# if __name__ == '__main__':
# queue = Queue(10)
# my_producer = Process(target=producer, args=(queue,))
# my_consumer = Process(target=consumer, args=(queue,))
# my_producer.start()
# my_consumer.start()
# my_consumer.join()
# my_producer.join()
# 共享全局变量通信
# 共享全局变量不适用于多进程编程，可以适用于多线程
# 多进程数据完全隔离
# from multiprocessing import Process, Queue
# import time
#
# def producer(a):
# a += 1
# time.sleep(2)
#
# def consumer(a):
# time.sleep(2)
# print(a)
#
# if __name__ == '__main__':
# a = 1
# my_producer = Process(target=producer, args=(a,))
# my_consumer = Process(target=consumer, args=(a,))
# my_producer.start()
# my_consumer.start()
# my_consumer.join()
# my_producer.join()
# multiprocessing 中的queue 不能用于pool进程池
# pool 中进程间通信需要使用manager中的queue
# from queue import Queue # 1
#
# from multiprocessing import Queue # 2
#
# from multiprocessing import Manager
# Manager().Queue() # 3
# from multiprocessing import Process, Queue, Pool, Manager
# import time
#
# def producer(queue):
# queue.put("a")
# time.sleep(2)
#
# def consumer(queue):
# time.sleep(2)
# data = queue.get()
# print(data)
#
# if __name__ == '__main__':
# queue = Manager().Queue(10)
# pool = Pool(2)
# pool.apply_async(producer, args=(queue,))
# pool.apply_async(consumer, args=(queue,))
# pool.close()
# pool.join()
# pipe 实现进程通信
# pipe 性能高于 queue
# from multiprocessing import Process, Queue, Pool, Manager, Pipe
# import time
#
# def producer(pipe):
# pipe.send("mojo")
#
# def consumer(pipe):
# print(pipe.recv())
#
# if __name__ == '__main__':
# recevie_pipe, send_pipe = Pipe()
# # pipe 只能适用2个指定的进程
# my_producer = Process(target=producer, args=(send_pipe,))
# my_consumer = Process(target=consumer, args=(recevie_pipe,))
# my_producer.start()
# my_consumer.start()
# my_producer.join()
# my_consumer.join()
from multiprocessing import Process, Queue, Pool, Manager
def add_data(p_dict, key, value):
p_dict[key] = value
if __name__ == '__main__':
# 内存共享
progress_dict = Manager().dict()
first_progress = Process(target=add_data, args=(progress_dict, "mojo1","22"))
second_progress = Process(target=add_data, args=(progress_dict, "mojo2","23"))
first_progress.start()
second_progress.start()
first_progress.join()
second_progress.join()
print(progress_dict)
11-12 本章小结
第12章 协程和异步io
本章节是一个过渡章节，也是从生成器过渡到协程的最重要的章节，本章节我们将从阻塞和非阻塞等概念开始一直到引出多线程和多进程编程在并发编程中的不足、IO多路复用，然后我们会通过事件循环+回调的方式完成高并发的请求，之后我们会讲解回调之痛以及生成器进阶中的 send、close 和 yield from 等功能，最后通过这...
12-1 并发、并行、同步、异步、阻塞、非阻塞
12-2 IO 多路复用 (select、poll 和 epoll)
12-3 select+回调+事件循环获取html-1
12-4 select+回调+事件循环获取html-2
12-5 回调之痛
12-6 协程是什么
12-7 生成器进阶-send、close和throw方法
12-8 生成器进阶-yield from-1
12-9 生成器进阶-yield from-2
12-10 生成器实现协程
12-11 async和await
12-12 本章小节
第13章 asyncio并发编程
asyncio 作为 python 未来最有野心也是最有前景的模块，是我们学习 python 高并发编程的必学模块。有了12章的基础，我们直接使用 asyncio 来进行并发编程就会变得容易理解，我们从 asyncio 的基本功能开始讲解、如何将任务提交到asyncio、如何将 ThreadPoolExecutor 和 asyncio 集成，明白 asyncio 内部是如...
13-1 事件循环-1
13-2 事件循环-2
13-3 task取消和子协程调用原理
13-4 call_soon、call_at、call_later、call_soon_threadsafe
13-5 ThreadPollExecutor 和 asycio 完成阻塞 IO 请求
13-6 asyncio 模拟 http 请求
13-7 future 和 task
13-8 asyncio同步和通信
13-9 aiohttp实现高并发爬虫 - 1
13-10 aiohttp实现高并发爬虫 - 2
13-11 aiohttp实现高并发爬虫 - 3
13-12 本章小节
第14章 课程总结
本章节我们会对课程的内容做一个整体的总结，加深大家对所学知识点的整体理解。
14-1 课程总结
185
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
