Data Analytics of an Information System Based on a Markov Decision Process and a Partially Observable Markov Decision Process

Data Analytics of an Information System Based on a Markov Decision Process and a Partially Observable Markov Decision Process

DOI: 

https://doi.org/10.30564/jcsr.v5i1.5434

Abstract

Data analytics of an information system is conducted based on a Markov decision process (MDP) and a partially observable Markov decision process (POMDP) in this paper. Data analytics over a finite planning horizon and an infinite planning horizon for a discounted MDP is performed, respectively. Value iteration (VI), policy iteration (PI), and Q-learning are utilized in the data analytics for a discounted MDP over an infinite planning horizon to evaluate the validity of the MDP model. The optimal policy to minimize the total expected cost of states of the information system is obtained based on the MDP. In the analytics for a discounted POMDP over an infinite planning horizon of the information system, the effects of various parameters on the total expected cost of the information system are studied.

Keywords: 

Predictive modelling, Information system, MDP, POMDP, Cybersecurity, Q-learning

Comments

Post a Comment

Popular posts from this blog

Experimental Simulation of Red Sprites in a Laboratory

Image
Experimental Simulation of Red Sprites in a Laboratory DOI:  https://doi.org/10.30564/jasr.v5i3.4858 Abstract Over the past three decades, research of high-altitude atmospheric discharges has received a lot of attention. This paper presents the results of experimental modeling of red sprites during a discharge in low-pressure air. To initiate ionization waves in a quartz tube, an electrodeless pulseperiodic discharge fed by microsecond voltage pulses with an amplitude of a few kilovolts and a repetition rate of tens of kHz were formed. In this case ionization waves (streamers) have a length of tens of centimeters. The main plasma parameters were measured at various distances along the tube. The measurements confirm the fact that ionization waves propagate in opposite directions from the zone of the main electrodeless discharge, just as it happens during the formation of red sprites. Keywords:  Red sprites, Experimental modeling, Streamer discharge in air, Low pressures
Read more

Analysis of Chinese Citizens’ Perception and Its Differences of City Spirit: A Case Study of Hefei City

Image
Analysis of Chinese Citizens’ Perception and Its Differences of City Spirit: A Case Study of Hefei City DOI:  https://doi.org/10.30564/jgr.v5i3.4788 Abstract City spirit is the soul of the city. The spread of city spirit not only could establish a civilized specimen for citizens, but also create a good cultural atmosphere for the city. Hefei residents’ perception of city spirit is extensive, and most of Hefei citizens consider the expression words of city spirit are very appropriate, which is enlightened, open-minded, actual and innovate. Great majority of Hefei citizens willing to support the propaganda and promotion of the city spirit, and they think the promotion of the city spirit plays a key role in the way of city’s development. In addition,significant differences in the perception of urban spirituality emerge among residents with different economic and social characteristics. There are four aspects about how to enhance the public perception of city spirit, which including in...
Read more

Bond Behavior between Recycled Concrete Containing CDW and Different Types of Steel Bars

Image
Bond Behavior between Recycled Concrete Containing CDW and Different Types of Steel Bars Abstract The operation of reinforced concrete structures is directly associated with the adhesion between the steel bar and the concrete, which allows the internal forces to be transferred to the reinforcement during the process of loading the structural elements. The modification of the concrete composition, with the introduction of recycled aggregate from construction and demolition waste (CDW), affects the steel-concrete interface and can modify the bonding stress, which is also influenced by the type and diameter of the bar used. In this work, the influence of the recycled fine aggregate (RFA) and types of steel bar on the steel-concrete bond was experimentally evaluated using the pullout test. Conventional concrete and recycled concrete, with RFA replacement level of 25%, were produced. Two types of steel rebars (i.e.,plain and deformed) with diameters of 10.0 and 16.0 mm were considered in th...
Read more