Neo4j通过json数据进行数据迁移

本地Neo4j数据库与服务器Neo4j数据库相差的只是ip地址，如果自己有插入所有节点和关系的代码的情况下更换一下ip地址和账密即可，如若没有源码或者部分节点是手动构建的，如何转移数据就成了问题。笔者通过自带的export json功能进行数据的迁移。

节点与关系导出

分布匹配所有的关系与节点，导出json格式数据，这里限制了显示上限为300，实际导出的文件是包含全部数据的，分别导出node.json与relation.json

数据迁移

使用python的json库导入json数据，同时进行一些违规字符的处理，之后先将节点导入数据库，同时插入旧的id值方便匹配，再将关系插入，最后删除旧id值，即可完成。速度较慢，可以考虑多线程插入提高转移速度。

import json
from py2neo import Graph, Node, Relationship, NodeMatcher


def get_graph():
    """
    connect the neo4j
    :return: neo4j object
    """
    try:
        graph = Graph("bolt://ip_addresss:7687", username='username', password='pwd')
        print("success for neo4j connection.")
        return graph
    except Exception as e:
        print(e)
        return None


def deal_json(file_name):
    """
    deal with the json
    :param file_name: json file name
    :return: list abound with dicts
    """
    with open(file_name, 'r', encoding='utf-8') as f:
        content = f.read()
        # necessarily, the beginning of file is an illegal character for utf-8
        if content.startswith(u'\ufeff'):
            content = content.encode('utf8')[3:].decode('utf8')
        # add other deal steps
        content = str(content).replace('	', '')
        load_dict = json.loads(content)
    return load_dict


def trans_nodes(graph, file_name):
    """
    transfer the nodes to new db
    :param graph: neo4j object
    :param file_name: json file name
    :return: None
    """
    load_dict = deal_json(file_name)
    for row in load_dict:
        # select by json structure
        old_id = row['n']['identity']
        node_type = row['n']['labels'][0]
        properties = row['n']['properties']
        node = Node(node_type, old_id=old_id)
        for key, val in properties.items():
            node[key] = val
        print('creating-', node)
        graph.create(node)


def trans_relations(graph, file_name):
    """
    transfer the relations to the new db
    :param graph: neo4j object
    :param file_name: json file name
    :return: None
    """
    load_dict = deal_json(file_name)
    mather = NodeMatcher(graph)
    index = 1
    for row in load_dict:
        start_id = row['p']['segments'][0]['start']['identity']
        start_label = row['p']['segments'][0]['start']['labels'][0]
        end_id = row['p']['segments'][0]['end']['identity']
        end_label = row['p']['segments'][0]['end']['labels'][0]
        relation_type = row['p']['segments'][0]['relationship']['type']
        start_node = mather.match(start_label, old_id=start_id).first()
        end_node = mather.match(end_label, old_id=end_id).first()
        relation = Relationship(start_node, relation_type, end_node)
        print('inserting ' + str(index) + '-', relation)
        index += 1
        graph.create(relation)


if __name__ == '__main__':
    graph = get_graph()
    graph.delete_all()
    trans_nodes(graph, './records_nodes.json')
    trans_relations(graph, './records_relations.json')
    # delete old id
    graph.run('MATCH (n) REMOVE n.old_id')