A general stochastic model to handle deduplication challenges using hidden Markov model in big data analytics
Downloads
Published
DOI:
https://doi.org/10.58414/SCIENTIFICTEMPER.2023.14.4.50Keywords:
Hidden markov model, Markov chain transition, Likelihood estimation, Poisson distribution.Dimensions Badge
Issue
Section
License
Copyright (c) 2023 The Scientific Temper
![Creative Commons License](http://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Background: Since increased interest of consumers, cloud computing is needed to store and access the information about their data in their convenient way. In recent days, cloud computing offers many services stipulated by the internet. Data duplication is one of the main challenges in big data analytics that leads to increased data storage and processing time. Therefore, there is a need to develop a data deduplication process. It eliminates excessive copies of data as well as decreases the storage space. In order to preserve the accurate data information without any duplication, joint probability distribution computes the likelihood of two events occurring together at the same time and thus it leads to removing the redundant data before data is sent to the cloud server.Abstract
Methods: this paper presents a GSM algorithm that uses hidden markov model, likelihood estimation, markov chain transition, and poisson distribution model.
Findings: Joint probability distribution computes the likelihood of two events occurring together at the same time and thus it leads to removing the redundant data before data is sent to the cloud server.
Novelty and applications: This paper proposes the general stochastic model (GSM) to handle redundant data by a multi-level process using hidden markov model (HMM), likelihood estimation, transition probability and poisson distribution model (PDM).
How to Cite
Downloads