• 中文核心期刊要目总览
  • 中国科技核心期刊
  • 中国科学引文数据库(CSCD)
  • 中国科技论文与引文数据库(CSTPCD)
  • 中国学术期刊文摘数据库(CSAD)
  • 中国学术期刊(网络版)(CNKI)
  • 中文科技期刊数据库
  • 万方数据知识服务平台
  • 中国超星期刊域出版平台
  • 国家科技学术期刊开放平台
  • 荷兰文摘与引文数据库(SCOPUS)
  • 日本科学技术振兴机构数据库(JST)

基于Cassandra的海量MUSER数据分布式存储与检索研究

The Study of the Data Storage and Retrieval for the Massive Data of MUSER Based on Cassandra

  • 摘要: 明安图射电频谱日像仪每天产生海量观测数据,传统的关系型数据库存储和管理这些数据时,面临着读写延迟高、性能和容量扩展能力有限以及可用性弱等诸多问题。针对这些问题,开展了基于NoSQL的海量数据存储与检索应用研究。首先,详细分析了明安图射电频谱日像仪的数据特点、存储需要以及面临的问题;然后,对明安图射电频谱日像仪进行数据建模,给出了明安图射电频谱日像仪的列式非关系型数据模型,同时提出了元数据和数据在NoSQL中的同步存储方法,解决了二者的一致性问题,在此基础上实现了基于Cassandra的海量天文数据存储管理系统(MBDMS);最后通过实验验证了系统存储与检索的高效性、扩展性以及可行性。实验结果表明,MBDMS可以很好地满足数据管理的需要,是解决当前数据存储问题的一种有效方案。

     

    Abstract: Mingantu Ultrawide Spectral Radioheliograph produces massive observational data every day. When the data is stored and managed by traditional relational database, problems appear, such as high latency in reading and writing, limited expansion capacity in performance and storage, and weak usability. In order to solve the problems, in this paper, researches on the application of MUSER data storage based on NoSQL has been carried out. First of all, we thoroughly analyze the characteristics of MUSER data, the requirement of storage and relating problems in detail. Secondly, the MUSER data model is given as well as the MUSER column non-relational data model. The metadata and data are synchronized in NoSQL based on Cassandra. The MUSER massive astronomical data storage management system (MBDMS) is realized based on Cassandra. The experiment validates the efficiency, expansibility and feasibility of data storage and retrieval of the system. As a result, MBDMS can well meet MUSER data management needs, is an effective program to solve the current MUSER data storage problem.

     

/

返回文章
返回