• 中文核心期刊要目总览
  • 中国科技核心期刊
  • 中国科学引文数据库(CSCD)
  • 中国科技论文与引文数据库(CSTPCD)
  • 中国学术期刊文摘数据库(CSAD)
  • 中国学术期刊(网络版)(CNKI)
  • 中文科技期刊数据库
  • 万方数据知识服务平台
  • 中国超星期刊域出版平台
  • 国家科技学术期刊开放平台
  • 荷兰文摘与引文数据库(SCOPUS)
  • 日本科学技术振兴机构数据库(JST)

Automation and parallelization scheme to accelerate pulsar observation data processing

  • Abstract: Previous studies aiming to accelerate data processing have focused on enhancement algorithms, using the graphics processing unit (GPU) to speed up programs, and thread-level parallelism. These methods overlook maximizing the utilization of existing central processing unit (CPU) resources and reducing human and computational time costs via process automation. Accordingly, this paper proposes a scheme, called SSM, that combines “Srun job submission mode”, “Sbatch job submission mode”, and “Monitor function”. The SSM scheme includes three main modules: data management, command management, and resource management. Its core innovations are command splitting and parallel execution. The results show that this method effectively improves CPU utilization and reduces the time required for data processing. In terms of CPU utilization, the average value of this scheme is 89%. In contrast, the average CPU utilizations of “Srun job submission mode” and “Sbatch job submission mode” are significantly lower, at 43% and 52%, respectively. In terms of the data-processing time, SSM testing on the Five-hundred-meter Aperture Spherical radio Telescope (FAST) data requires only 5.5 h, compared with 8 h in the “Srun job submission mode” and 14 h in the “Sbatch job submission mode”. In addition, tests on the FAST and Parkes datasets demonstrate the universality of the SSM scheme, which can process data from different telescopes. The compatibility of the SSM scheme for pulsar searches is verified using 2 days of observational data from the globular cluster M2, with the scheme successfully discovering all published pulsars in M2.

     

/

返回文章
返回