In this post will give introduction to Markov models and Hidden Markov models as mathematical abstractions, with some examples.


In probability theory, a Markov model is a stochastic model that assumes the Markov property. A stochastic model models a process where the state depends on previous states in a non-deterministic way. A stochastic process has the Markov property if the conditional probability distribution of future states of the process.

 


System state is fully observable System state is partially observable

System is autonomous

Markov chain Hidden Markov model

System is controlled
Markov decision process Partially observable Markov decision process

 

Markov chain

A Markov chain named by Andrey Markov, is a mathematical system that representing transitions from one state to another on a state space. The state is directly visible to the observer. It is a random process usually characterized as memoryless: the next state depends only on the current state and not on the sequence of events that preceded it. This specific kind of "memorylessness" is called the Markov property.

 

Let's talk about the weather. we have three types of weather sunny, rainy and cloudy.

Let's assume for the moment that the weather lasts all day and it does not change from rainy to sunny in the middle of the day.

Weather prediction is try to guess what the weather will be like tomorrow based on a history of observations of weather

simplified model of weather prediction
Wewill collect statistics on what the weather was like today based on what the weather was like yesterday the day before and so forth. We want to collect the following probabilities.

 

image

Using above expression, we can give probabilities of types of weather for tomorrow and the next day using n days of history.

image

 

The larger n will be problem in here. The more statistics we must collect Suppose that n=5 then we must collect statistics for 35 = 243 past histories Therefore we will make a simplifying assumption called the "Markov Assumption".

image

This is called a "first-order Markov assumption" since we say that the probability of an observation at time n only depends on the observation at time n-1. A second-order Markov assumption would have the observation at time n depend on n-1 and n-2. We can the express the joint probability using the “Markov assumption”.

image

So this now has a profound affect on the number of histories that we have to find statistics for, we now only need 32 = 9 numbers to characterize the probabilities of all of the sequences. (This assumption may or may not be a valid assumption depending on the situation.)

Arbitrarily pick some numbers for  P (wtomorrow | wtoday).

 

image

Tabel2: Probabilities of Tomorrow's weather based on Today's Weather

 

“What is w0?” In general, one can think of w as the START word so P(w1w2) is the probability that w1 can start a sentence.

For first-order Markov models we can use these probabilities to draw a probabilistic finite state automaton.

image

eg:

1. Today is sunny what's the probability that tomorrow is sunny and the day after is rainy

First we translates into

  • P(w2= sunny,w3=rainy|w1=sunny)
P(w2= sunny,w3=rainy|w1=sunny) = P(w2=sunny|w1=sunny) *
   P(w3=rainy|w2=sunny,w3sunny)
  = P(w2=sunny|w1=sunny) * P(w3=rainy|w2=sunny)
  = 0.8 * 0.05
  =0.04

 

Hidden Markov model

A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved (hidden) states. A HMM can be considered the simplest dynamic Bayesian network. Hidden Markov models are especially known for their application in temporal pattern recognition such as speech, handwriting, gesture recognition, part-of-speech tagging, musical score following, partial discharges and bioinformatics.

Example

Well suppose you were locked in a room for several days and you were asked about the weather outside The only piece of evidence you have is whether the person who comes into the room carrying your daily meal is carrying an umbrella or not.

image

Table 3: Probabilities of Seeing an Umbrella

The equation for the weather Markov process before you were locked in the room.
image

 

Now we have to factor in the fact that the actual weather is hidden from you We do that by using Bayes Rule.

image

Where ui is true if your caretaker brought an umbrella on day i and false if the caretaker did not. The probability P(w1,..,wn) is the same as the Markov model from the last section and the probability P(u1,..,un) is the prior probability of seeing a particular sequence of umbrella events.

The probability P(w1,..,wn|u1,..,un) can be estimated as,

image

Assume that for all i given wi, ui is independent of all uj and wj. I and J not equal

Next post will explain about “Markov decision process” and “Partially observable Markov decision process”.

0

Add a comment

We used have  Singleton Design Pattern in our applications whenever it is needed. As we know that in singleton design pattern we can create only one instance and can access in the whole application. But in some cases, it will break the singleton behavior.

13

Microservices can have a positive impact on your enterprise. Therefore it is worth to know that, how to handle Microservice Architecture (MSA) and some Design Patterns for Microservices. General goals or principles for a microservice architecture.

12

Last few years has been a great year for API Gateways and API companies. APIs (Application Programming Interfaces) are allowing businesses to expand beyond their enterprise boundaries to drive revenue through new business models.

1

kubectl (Kubernetes command-line tool) is to deploy and manage applications on Kubernetes. Using kubectl, you can inspect cluster resources; create, delete, and update components.

NOTE

You must use a kubectl version that is within one minor version difference of your cluster.

2

WSO2 Enterprise Integrator is shipped with a separate message broker profile (WSO2 MB). In this Post I will be using message broker profile in EI (6.3.0).

1

When two or many applications want to exchange data, they do so by sending the data through a channel that connects the each others.

2

Microservices are going completely over the enterprise and changed the way people write software within an enterprise ecosystem.

Let build you microservices with msf4j for Auto Mobile.

The SMPP inbound endpoint allows you to consume messages from SMSC via WSO2 ESB OR EI.

1.  Start SMSC

2.  Create custom inbound end point with below parameter. (Make sure you pick correct system-id and password correct for your SMSC)

3. Create Sequence for Inbound EP.

4. Once ESB or EI start.

1

WSO2 APIM Components

WSO2 API Manager includes five main components as the Publisher, Store, Gateway, Traffic Manager and Key Manager.

API Gateway - responsible for securing, protecting, managing, and scaling API calls.

There is REST Back-End end-point in Vehicle registration services as below

GET /car?name=prius HTTP/1.1

Host: localhost:8080

color: White

Company need to expose it 3rd part companies and above End-point should not change as internal services are using it.

1

Estimation for Software project development is the process of predicting the most realistic amount of effort (expressed in terms of person-hours or money) required to develop or maintain software based on incomplete, uncertain and noisy input.

1

In this post give some basic on JAVA Stream API which is added in Java 8. It works very well in conjunction with lambda expressions. Pipeline of stream operations can manipulate data by performing operations like search, filter, count, sort, etc.

The Lifecycle Management(LCM) plays a major role in SOA Governance. WSO2 Governance Registry Lifecycle Management supports access control at multiple levels in lifecycle state.

1.

1

Enterprise Data Integration is a broad term used in the integration landscape to connect multiple Enterprise  applications and hardware systems within an organization. All these enterprise data integration lead to achieve to remove the complexity by simplifying data management as a whole.

Post is very basic one, Since Talend is all about data integration. Finding a BigDecimal [1] in such data set is very common.

BigDecimal VS Doubles

A BigDecimal is an exact way of representing numbers. A Double has a certain precision.

Vehicles registration services using REST services on government TAX department system. That REST services give the TAX information for the Vehicle.

{"Tax": {"Amount": 58963}}

Vehicles registration Depart planning to extend the service and expose as below.

1

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. Today some systems may store data in a denormalized form and data integration tools able handle those.

There is few thing that make my work enjoyable with WSO2 ESB as it provides support for JavaScript Object Notation (JSON) payloads in messages. It is not very new feature and it old feature.

Working on an Alienvault IDS system or OSSIM you can come across over huge amount of alarms are created will system migrations.

If you’re familiar with SEIM tools or OSSEC, then you know syscheck. Syscheck is the integrity checking daemon within OSSEC. It’s purpose is simple, identify and report on changes within the system files.

Triggering action over the event occurrence in OSSIM is going to explain in this article.

There is agent in the system with IP, 192.168.80.22. Email is to be send to server admins whenever this agent disconnect and reconnect to SEIM server.

We need to have extra user data field on our security event. We need to know

event occurred time Host Server IP Editing particular event on ‘/etc/ossim/agent/plugins/ossec-single-line.cfg’. We can achieve it. We are interest on Web group and ID 0030. We added below line as our need.

Pre request

Test OSSEC new log from ‘ossec-logtest’

Here is the custom created rules.

In here I am using well known decoder in OSSEC if you need new OSSEC decoder you can write new decoder also [1]. Add new file to  rules directory in OSSEC.

Creating new OSSEC rule set

$ vi var/ossec/rules/custom_access_rules.xml

In here I am interest to monitor web user behavior model.

1

Introductions

In OSSEC, the rules are classified in multiple levels from the lowest (00) to the maximum level 16. But some levels are not used right now and below explain level details.

A brute-force attack consists of an attacker trying many passwords or passphrases with the hope of eventually guessing correctly. The attacker systematically checks all possible passwords and passphrases until the correct one is found.

Unfortunately Windows does not support Fdisk anymore. But there is another good command line tool to solve this problem. DiskPart in windows is useful format unallocated spaces in USB pen.

1. Enter ‘diskpart’ in cmd

Then disk part will start

2. List down storage in PC by

list disk

3.

The Linux kernel in Ubuntu provides a packet filtering system called netfilter, and the traditional interface for manipulating netfilter are the iptables suite of commands. The Uncomplicated Firewall (ufw) is a frontend for iptables and is particularly well-suited for host-based firewalls.

Count line when words has been matched

$ grep -c 'word' /path/to/file

Pass the -n option to precede each line of output with the number of the line in the text file

$ grep -n 'root' /etc/passwd

Ignore word case

$ grep -i 'word' /path/to/file

Use grep recursively under each directory

$ grep -r

Each application contains it's own log record format.

Access log moves to sensor / data source then I mapping to event id with considering the rules in ossim.

Data sources can be found in “ossim ->configuration –> threat_intelligence –> data_source” and search for source as below. Pick “AlienVault HIDS-accesslog” and it reads the access log.

It provides the SSH authentication to the host you want to access. For Cisco devices (PIX, routers, etc), you need to provide an additional parameter for the enable password. The same thing applies if you want to add support for “su”, it must be the additional parameter.

1. Log into AlienVault USM.

1. Download the image file of OSSIM

2. Make bootable pen with OSSIM ISO file

3. Boot drive

Make sure you have internet connection

4. Select OSSIM server to install

5. Just follow the the wizard

6. Add the net work details correctly with unique new IP for OSSIM server.

7.

Finding the logs in my server. I generally use lsof to list what is my server.

lsof | grep log

I check which log are reading by OSSEC

Check cat /var/ossec/etc/ossec.conf  |grep "<location>/"

Add new access log to OSSCE.

I am
I am
Archives
Total Pageviews
Total Pageviews
2 0 5 8 0 4 9
Categories
Categories
Loading