Data Analytics of an Information System Based on a Markov Decision Process and a Partially Observable Markov Decision Process

Data Analytics of an Information System Based on a Markov Decision Process and a Partially Observable Markov Decision Process

DOI: 

https://doi.org/10.30564/jcsr.v5i1.5434

Abstract

Data analytics of an information system is conducted based on a Markov decision process (MDP) and a partially observable Markov decision process (POMDP) in this paper. Data analytics over a finite planning horizon and an infinite planning horizon for a discounted MDP is performed, respectively. Value iteration (VI), policy iteration (PI), and Q-learning are utilized in the data analytics for a discounted MDP over an infinite planning horizon to evaluate the validity of the MDP model. The optimal policy to minimize the total expected cost of states of the information system is obtained based on the MDP. In the analytics for a discounted POMDP over an infinite planning horizon of the information system, the effects of various parameters on the total expected cost of the information system are studied.

Keywords: 

Predictive modelling, Information system, MDP, POMDP, Cybersecurity, Q-learning

Comments

Post a Comment

Popular posts from this blog

Impact of Polymer Coating on the Flexural Strength and Deflection Characteristics of Fiber-Reinforced Concrete Beams

Image
Impact of Polymer Coating on the Flexural Strength and Deflection Characteristics of Fiber-Reinforced Concrete Beams DOI:  https://doi.org/10.30564/jbms.v4i2.5304 Abstract Liquid polymers (LP) have become an important structural material used in the construction industry in the last decade. This paper investigates the viability of using commercially available LPs as a coating material to improve the flexural strength of fiber-modified concrete beams. The scope included preparing rectangular prism concrete beams with a concrete mixture including fly ash and fiber and coating them with four different liquid polymers at a uniform thickness following the curing process while one set of samples was maintained under the same conditions as a control group without coating. In addition, cylindrical samples were prepared to determine the compressive strength of the concrete mixture. Following the curing process in an unconfined open-air laboratory environment for another 28 days, concrete sample
Read more

Achieving Sustainable Use and Management of Water Resources for Irrigation in Nigeria

Image
Achieving Sustainable Use and Management of Water Resources for Irrigation in Nigeria DOI:  https://doi.org/10.30564/jees.v2i2.2505 Abstract Water resource is available in Nigeria to meet water demand for domestic, agricultural and industrial use. If well enhanced and protected from various source of pollution. However, less information is available on suitability of water resources for irrigation use in Nigeria. This article reviews literature of the past with regards to influence of geogenic and anthropogenic activities on water resource for irrigation purpose in Nigeria and also to explain the current state of suitability of water resource for irrigation studies in Nigeria and gaps in studies. It also summarizes future ways on water resource management and preventive measure for water resource pollution for irrigation use. Related articles were downloaded from Google scholar in water-related issues. This paper tends to review previous article on water resource in Nigeria, and its su
Read more

Soil Bunds Effect on Soil Properties under Different Topographies of the Southwest Ethiopia

Image
Soil Bunds Effect on Soil Properties under Different Topographies of the Southwest Ethiopia Abstract Soil erosion is a major environmental problem affecting development endeavor. Physical soil and water conservation (SWC) measures such as soil bunds are implemented to mitigate soil erosion. However, information on the effects of soil bunds on soil fertility is limited. This study was aimed to evaluate soil quality in fields with soil bunds and with no soil bunds in steep, middle and lower sloping cultivated lands as well as spatial variation of soil properties in between bunds in southwest Ethiopia. About 7-15 years old bunds and nearby cultivated fields lacking bunds were assessed. From 0 cm-20 cm soil depth, a 36 soil samples were collected. Soil texture, soil organic carbon (SOC), total nitrogen (N tot ) and exchangeable potassium (K exch ) were analyzed. Soil bunds showed significantly ( p <0.0.5) greater clay but less sand than adjacent no-bund fields. In steep, middle, and low
Read more