优达学城python项目P1：搜索和探索近地天体(NEOs)

项目1官方地址：https://github.com/udacity/nd303-c1-advanced-python-techniques-project-starter

1 概述

本项目概言之，实现一个命令行工具来检查和查询近地天体的数据集。

完成后，您将能够检查数据集中的近地天体的属性，并使用以下任意组合的过滤器查询近地天体的数据集:

• 发生在给定的日期。
• 在给定的开始日期或之后发生。
• 在给定的结束日期或之前发生。
• 以至少(或至多)X天文单位的距离接近地球。
• 以至少(或最多)Y千米每秒的相对速度接近地球。
• 直径至少等于(或至少小于)Z公里。
• 被美国宇航局标记为有潜在危险(或没有)。

学习目标

通过完成本项目，您将展示以下能力：

• 用Python表示结构化数据。
• 将结构化文件中的数据提取到Python中。
• 根据所需的行为在Python中转换数据。
• 以结构化的方式将结果保存到文件中。

在此过程中，您必须能够：

• 编写Python函数来转换数据和执行算法。
• 设计Python类来封装有用的数据类型。
• 为复杂的实现提供接口抽象。

在这个过程中遇到bug是很正常的，所以很有可能，您还将获得宝贵的调试技能的练习，无论是解释堆栈跟踪、跟踪系统错误、处理和抛出适当的错误、使用pdb遍历代码、使用assert检查先决条件，还是仅仅使用print显示内部状态。

 

具体任务

• 任务0:检查数据。(数据/ neos.csv和数据/ cad.json)
• 任务1:构建模型来表示数据。(models.py)
• 任务2:将数据提取到自定义数据库中(Extract.py和database.py)
• 任务3:创建过滤器来查询数据库以生成匹配的close - approach对象流，并限制结果大小。(filters.py和database.py)
• 任务4:将数据保存到文件中。(write.py)

 

技能大模块：

• 单元测试Python3 unittest
  • 类的测试
  • 文件内容测试
  • python版本测试
  • 文件存在否测试
  • 文件格式测试
  • 。。。
•  

 

 

2 项目结构

.
├── README.md       # This file.
├── main.py
├── models.py       # Task 1.
├── read.py         # Task 2a.
├── database.py     # Task 2b and Task 3b.
├── filters.py      # Task 3a and Task 3c.
├── write.py        # Task 4.
├── helpers.py
├── data
    ├── neos.csv
    └── cad.json

 

Task 1：设计存储数据的两个类

from helpers import cd_to_datetime, datetime_to_str

class NearEarthObject:
    """近地天体的基本信息类，处理neos.csv文件
    """
    def __init__(self, **info):
        """参数 info: 一个字典.
        例子：
        dict_info = {'designation':'编号GT520', 'time':'哈雷彗星', 'diameter':101, 'hazardous':'Y'}
        neo = NearEarthObject(**dict_info)
        """
        self.designation = (str(info['designation']) if info['designation'] else '')
        self.name = str(info['name']) if info['name'] else None
        self.diameter = (float(info['diameter']) if info['diameter'] else float('nan'))
        self.hazardous = True if info['hazardous'] == 'Y' else False

        self.approaches = []

    #加了@property后，可以用调用属性的形式来调用方法,后面不需要加（）
    @property
    def fullname(self):
        return f"{self.designation} ({self.name or 'N/A'})"

    #print(类的实例名)时，会调用此方法。功能相当于此实例的“自我介绍”。
    def __str__(self):
        return (f"""{self.fullname} 是一个直径为 """
                f"""{self.diameter:.3f}km 且 """
                f"""{'具有' if self.hazardous else '不具有'} 危险的近地天体。""")

    # print(类的实例名)时，当类中没有__str__方法的时候，会去调用__repr__方法，所以一般类中需要定义repr方法
    # 也可以直接这样调用__repr__方法：print(repr(类的实例名))
    # 如果没有自定义__repr__方法和__str__方法，print(类的实例名)时输出默认的__repr__方法：“类名+object at+内存地址”
    def __repr__(self):
        return (f"NearEarthObject(designation={self.designation!r}, name={self.name!r}, "
                f"diameter={self.diameter:.3f}, hazardous={self.hazardous!r})")



class CloseApproach:
    """临近地球时的状态类，处理cad.json文件
    """
    def __init__(self, **info):
        """参数 info: 一个字典.
        """
        self._designation = (str(info['designation']) if info['designation']
                             else '')
        self.time = cd_to_datetime(info['time']) if info['time'] else None
        self.distance = float(info['distance']) if info['distance'] else 0.0
        self.velocity = float(info['velocity']) if info['velocity'] else 0.0

        # Create an attribute for the referenced NEO, originally None.
        self.neo = None

    @property
    def fullname(self):
        """Return a representation of the full name of this NEO."""
        return f"{self._designation} ({self.neo.name or 'N/A'})"

    @property
    def time_str(self):
        return datetime_to_str(self.time)

    def __str__(self):
        return (f"""在 {self.time_str}时刻, '{self.fullname}' 将接近地球， """
                f"""最近距离为 {self.distance:.2f} au，速度为 """
                f"""{self.velocity:.2f} km/s.""")

    def __repr__(self):
        return (f"CloseApproach(time={self.time_str!r}, distance={self.distance:.2f}, "
                f"velocity={self.velocity:.2f}, neo={self.neo!r})")

NearEarthObject类的测试效果如下：

 

Task 2：从文件中提取数据放到Python类对象中

1 CSV和json文件数据提取函数

import csv
import json

from models import NearEarthObject, CloseApproach


def load_neos(neo_csv_path):
    """Read near-Earth object information from a CSV file.

    :param neo_csv_path: A path to a CSV file containing data about near-Earth objects.
    :return: A collection of `NearEarthObject`s.
    """
    # TODO: Load NEO data from the given CSV file.
    neos = []
    with open(neo_csv_path, 'r') as infile:
        reader = csv.DictReader(infile)
        for row in reader:
            neo = NearEarthObject(designation=row['pdes'],
                                  name=row['name'],
                                  diameter=row['diameter'],
                                  hazardous=row['pha'],)
            neos.append(neo)
    return neos


def load_approaches(cad_json_path):
    """Read close approach data from a JSON file.

    :param neo_csv_path: A path to a JSON file containing data about close approaches.
    :return: A collection of `CloseApproach`es.
    """
    # TODO: Load close approach data from the given JSON file.
    approaches = []

    with open(cad_json_path) as infile:
        content = json.load(infile)
        data = content['data']
        fields = {}

        for key in content['fields']:
            fields[key] = content['fields'].index(key)

        for row in data:
            approach = CloseApproach(designation=row[fields['des']],
                                     time=row[fields['cd']],
                                     distance=row[fields['dist']],
                                     velocity=row[fields['v_rel']],)
            approaches.append(approach)
    return approaches

2 

class NEODatabase:
    """一个包含near-Earth类及其近地时信息的数据库.

    A `NEODatabase` contains a collection of NEOs and a collection of close
    approaches. It additionally maintains a few auxiliary data structures to
    help fetch NEOs by primary designation or by name and to help speed up
    querying for close approaches that match criteria.
    """
    def __init__(self, neos, approaches):
        """Create a new `NEODatabase`.

        As a precondition, this constructor assumes that the collections of NEOs
        and close approaches haven't yet been linked - that is, the
        `.approaches` attribute of each `NearEarthObject` resolves to an empty
        collection, and the `.neo` attribute of each `CloseApproach` is None.

        However, each `CloseApproach` has an attribute (`._designation`) that
        matches the `.designation` attribute of the corresponding NEO. This
        constructor modifies the supplied NEOs and close approaches to link them
        together - after it's done, the `.approaches` attribute of each NEO has
        a collection of that NEO's close approaches, and the `.neo` attribute of
        each close approach references the appropriate NEO.

        :param neos: A collection of `NearEarthObject`s.
        :param approaches: A collection of `CloseApproach`es.
        """
        self._neos = neos
        self._approaches = approaches
        self._data = {}

        for approach in self._approaches:
            if approach._designation not in self._data:
                self._data[approach._designation] = {
                    'approaches': [],
                    'neo': None,
                }
            self._data[approach._designation]['approaches'].append(approach)


        for neo in self._neos:
            if neo.designation not in self._data:
                self._data[neo.designation] = {
                    'approaches': [],
                    'neo': None,
                }
            self._data[neo.designation]['neo'] = neo

            if neo.name not in self._data:
                self._data[neo.name] = neo.designation

            neo.approaches.extend(self._data[neo.designation]['approaches'])


        for approach in self._approaches:
            approach.neo = self._data[approach._designation]['neo']




    def get_neo_by_designation(self, designation):
        """Find and return an NEO by its primary designation.

        If no match is found, return `None` instead.

        Each NEO in the data set has a unique primary designation, as a string.

        The matching is exact - check for spelling and capitalization if no
        match is found.

        :param designation: The primary designation of the NEO to search for.
        :return: The `NearEarthObject` with the desired primary designation, or `None`.
        """
        if designation in self._data:
            return self._data[designation]['neo']
        return None

    def get_neo_by_name(self, name):
        """Find and return an NEO by its name.

        If no match is found, return `None` instead.

        Not every NEO in the data set has a name. No NEOs are associated with
        the empty string nor with the `None` singleton.

        The matching is exact - check for spelling and capitalization if no
        match is found.

        :param name: The name, as a string, of the NEO to search for.
        :return: The `NearEarthObject` with the desired name, or `None`.
        """
        if name in self._data:
            return self.get_neo_by_designation(self._data[name])
        return None

    def query(self, filters=()):
        """Query close approaches to generate those that match a collection of filters.

        This generates a stream of `CloseApproach` objects that match all of the
        provided filters.

        If no arguments are provided, generate all known close approaches.

        The `CloseApproach` objects are generated in internal order, which isn't
        guaranteed to be sorted meaninfully, although is often sorted by time.

        :param filters: A collection of filters capturing user-specified criteria.
        :return: A stream of matching `CloseApproach` objects.
        """
        if filters:
            for approach in self._approaches:
                if all(map(lambda f: f(approach), filters)):
                    yield approach
        else:
            for approach in self._approaches:
                yield approach

 

Task 3：设计数据库条件查询方法

 

 

Task 4：保存查询结果

 

 

 

重要知识点：

1. @property作用：可以用调用属性的形式来调用方法（函数）,后面不需要加（）。

2. @classmethod作用：很多博客只是说@calssmethod的作用就是“可以不需要实例化，直接类名.方法名()来调用。这有利于组织代码，把某些应该属于某个类的函数给放到那个类里去，同时有利于命名空间的整洁”。很抽象，其实具体作用如下：

   @classmethod的作用实际是可以在class内实例化class，一般使用在有工厂模式要求时。作用就是比如输入的数据需要清洗一遍再实例化，可以把清洗函数定义在class内部并加上@classmethod装饰器已达到减少代码的目的。总结起来就是：@class method可以用来为一个类创建一些预处理的实例。https://blog.csdn.net/qq_23981335/article/details/103798741

3.