前言
随着数据量的爆炸性增长和业务需求的日益复杂化,传统的单一模型数据库已经难以满足复杂多变的业务需求。尽管当前针对不同的数据类型,例如关系型数据、文档数据、图数据和时序数据业内提供了多种数据库以应对存储及处理需求,但是在实际应用中,许多场景需要将多种数据模型结合起来,以支持复杂的业务逻辑和决策分析。
统一的多模型大数据平台的出现,为解决这一问题提供了创新的思路。它能够在单一平台上同时支持多种数据模型,从而减少了跨系统整合的复杂性和成本。在联合查询场景中,多模型数据库的能力尤为重要:它不仅可以高效处理不同模型之间的数据联动,还能够为用户提供统一的查询接口,显著提高数据访问效率和开发效率。
因此,数据库联合查询的能力显得尤为重要。
基础背景介绍
ArgoDB
Transwarp ArgoDB 是星环科技自主研发的分布式分析型数据库,可以替代 Hadoop+MPP 混合架构。支持标准 SQL 语法,提供多模分析、实时数据处理、存算解耦、混合负载、数据联邦、异构服务器混合部署等先进技术能力。ArgoDB 基于多模型统一架构,支持关系型、宽表、时序、地理空间、搜索、图、向量等市面主流的 11种数据模型。通过一个 ArgoDB 数据库,就可以满足 OLAP、AETP、多模型融合分析、联邦计算、数据仓库、实时数据仓库、湖仓集一体等场景。
更多案例分享及最佳实践详见:
- 【数据处理效率提升实践】ArgoDB如何助力企业全面实现数据处理效率最大化?
- 【最新案例】ArgoDB新功能之读写分离,助力某医药集团打造高效数据中心,消除传统方案的灵活性限制,确保响应时间的可预测性
- 【指标查询调优实践案例】ArgoDB助力某银行实现性能全面提升
- 【CDH国产化替代案例】全面简化架构,降低成本,大幅提升数据处理效率
StellarDB
Transwarp StellarDB 是一款为企业级图应用而打造的分布式图数据库,用于快速查找数据间的关联关系,并提供强大的算法分析能力。StellarDB克服了万亿级关联图数据存储的难题,通过自定义图存储格式和集群化存储, 实现了传统数据库无法提供的低延时多层关系查询,在社交网络、金融领域都有巨大应用潜力。
场景示例
基于社交网络的推荐系统
- 显式的社交网络关系已经存在于很多购物消费、休闲娱乐的产品平台中,这种社交网络的实体和实体间的关联关系都处于不断变化的过程中,比如关注列表的变化、兴趣点的变化、流行趋势的变化等等。可以使用 StellarDB 分布式图数据库存储客户的社交网络信息、购买历史记录,好友列表和感兴趣商品等关联关系数据。通过结合传统神经网络和图深度学习,融合用户兴趣爱好模型和社交网络影响,共同决定用户消费的推荐行为,从而提升用户推荐体验感。
反洗钱
- 通过关联查询、可视化图分析、图挖掘、机器学习和规则引擎的结合使用,可快速检索关联关系数据,并挖掘隐藏关系并模型化业务经验,帮助金融机构的建立一个可持续、经济可行的反洗钱合规框架。
反欺诈
- 结合征信、税务、诉讼、工商、企业业务系统数据等多种数据数据源类型,使用 StellarDB 构建并存储关联关系图谱,通过使用 StellarDB 内置图算法或查询功能获得图谱中图相关特征,如出入度、中心度等,基于社区发现、标签扩散等算法,融合机器学习模型,构建可疑团伙发现模型,识别风险客户和集团,降低反欺诈分析人力成本,提升效率。
于今年,社区开发版正式发布StellarDB图数据库,感兴趣的小伙伴们可以下载体验。
相关链接:
跨多种类数据库联合查询实践
实践目的
演示星环数据库产品的异构数据分析能力
演示结论
- 图数据处理能力支持
- 支持图数据与关系型数据联邦查询
演示步骤
1. 创建图模型存储的表
config query.lang cypher;
create graph kg_ship_order with schema(:Employee {lastName string,firstName string,title string,titleOfCourtesy string,birthDate localdatetime,hireDate localdatetime,address string,city string,region string,postalCode string,country string,homePhone string,extension long,photo string,notes string,reportsTo int,photoPath string}) (:Customer {companyName string,contactName string,contactTitle string,address string,city string,region string,postalCode string,country string,phone string,fax string}) (:Orders {orderDate localdatetime,requiredDate localdatetime,shippedDate localdatetime,shipVia int,freight double,shipName string,shipAddress string,shipCity string,shipRegion string,shipPostalCode string,shipCountry string}) [:Buy{orderDate localdatetime,requiredDate localdatetime,shippedDate localdatetime,shipVia int,freight double,shipName string,shipAddress string,shipCity string,shipRegion string,shipPostalCode string,shipCountry string}] [:SoldBy{orderDate localdatetime,requiredDate localdatetime,shippedDate localdatetime,shipVia int,freight double,shipName string,shipAddress string,shipCity string,shipRegion string,shipPostalCode string,shipCountry string}] graphproperties: {`graph.shard.number` : 1};
2. 通过SQL批量写入图模型数据
use graph kg_ship_order;
create (Employee1:Employee {__uid: "1" , lastName:"Davolio",firstName:"Nancy",title:"Sales Representative",titleOfCourtesy:"Ms.",birthDate:localdatetime("1948-12-08T00:00:00.000"),hireDate:localdatetime("1992-05-01T00:00:00.000"),address:"507 20th Ave. E. Apt. 2A",city:"Seattle",region:"WA",postalCode:"98122",country:"USA",homePhone:"(206) 555-9857",extension:5467,photo:"0x151C2F00020000000D000E0014002100FFFFFFFF4269746D617020496D616765005061696E742E506963747572650001050000020000000700000050427275736800000000000000000020540000424D20540000000000007600000028000000C0000000DF0000000100040000000000A0530000CE0E0000D80E0000000000",notes:"Education includes a BA in psychology from Colorado State University in 1970. She also completed The Art of the Cold Call. Nancy is a member of Toastmasters International.",reportsTo:2,photoPath:"http://accweb/emmployees/davolio.bmp" } ), (Employee2:Employee {__uid: "2" , lastName:"Fuller",firstName:"Andrew",title:"Vice President Sales",titleOfCourtesy:"Dr.",birthDate:localdatetime("1952-02-19T00:00:00.000"),hireDate:localdatetime("1992-08-14T00:00:00.000"),address:"908 W. Capital Way",city:"Tacoma",region:"WA",postalCode:"98401",country:"USA",homePhone:"(206) 555-9482",extension:3457,photo:"0x151C2F00020000000D000E0014002100FFFFFFFF4269746D617020496D616765005061696E742E506963747572650001050000020000000700000050427275736800000000000000000020540000424D20540000000000007600000028000000C0000000DF0000000100040000000000A0530000CE0E0000D80E0000000000",notes:"Andrew received his BTS commercial in 1974 and a Ph.D. in international marketing from the University of Dallas in 1981. He is fluent in French and Italian and reads German. He joined the company as a sales representative was promoted to sales manager",reportsTo:null,photoPath:"http://accweb/emmployees/fuller.bmp" } ), (Employee3:Employee {__uid: "3" , lastName:"Leverling",firstName:"Janet",title:"Sales Representative",titleOfCourtesy:"Ms.",birthDate:localdatetime("1963-08-30T00:00:00.000"),hireDate:localdatetime("1992-04-01T00:00:00.000"),address:"722 Moss Bay Blvd.",city:"Kirkland",region:"WA",postalCode:"98033",country:"USA",homePhone:"(206) 555-3412",extension:3355,photo:"0x151C2F00020000000D000E0014002100FFFFFFFF4269746D617020496D616765005061696E742E506963747572650001050000020000000700000050427275736800000000000000000080540000424D80540000000000007600000028000000C0000000E0000000010004000000000000540000CE0E0000D80E0000000000",notes:"Janet has a BS degree in chemistry from Boston College (1984). She has also completed a certificate program in food retailing management. Janet was hired as a sales associate in 1991 and promoted to sales representative in February 1992.",reportsTo:2,photoPath:"http://accweb/emmployees/leverling.bmp" } ), (Employee4:Employee {__uid: "4" , lastName:"Peacock",firstName:"Margaret",title:"Sales Representative",titleOfCourtesy:"Mrs.",birthDate:localdatetime("1937-09-19T00:00:00.000"),hireDate:localdatetime("1993-05-03T00:00:00.000"),address:"4110 Old Redmond Rd.",city:"Redmond",region:"WA",postalCode:"98052",country:"USA",homePhone:"(206) 555-8122",extension:5176,photo:"0x151C2F00020000000D000E0014002100FFFFFFFF4269746D617020496D616765005061696E742E506963747572650001050000020000000700000050427275736800000000000000000020540000424D20540000000000007600000028000000C0000000DF0000000100040000000000A0530000CE0E0000D80E0000000000",notes:"Margaret holds a BA in English literature from Concordia College (1958) and an