What is Helix?

It  used for the automatic management of partitioned, replicated and distributed resources hosted on a cluster of nodes. Helix automates reassignment of resources in the face of node failure and recovery, cluster expansion, and reconfiguration. Modeling a distributed system as a state machine with constraints on states and transitions.

Terminologies

  • Node :  A single machine
  • Cluster: Set of Nodes   
  • Resource : A logical entry (e.g.    database, index, task)
  • Partition: Subset of the resource  (Each subtask is referred to as a partition)
  • Replica: Copy of a Partition State  (e.g Master, Slave). It increase the availability of the system
  • State: Describes the role of a replica (Each node in the cluster has its own Current State)
  • State Machine and Transitions: An action that allows a replica to move from one state to another, thus changing its role. ( e.g    Slave    -->    Master )  
  • spectators: the external clients. Helix provides an External View that is an aggregated view of the current state across all nodes.
  • Current State: represents resource's actual state at a participating node.
    - INSTANCE_NAME: Unique name representing the process
    - SESSION_ID: ID that is automatically assigned every time a process joins the cluster
  • Rebalancer: The core component of Helix is the Controller which runs the Rebalance algorithm on every cluster event.
  • Dynamic Ideal State: Helix powerful is that Ideal State can be changed dynamically. It is adjusting the ideal state. Whenever a cluster event occurs, Helix can operate in one of three modes
  1. FULL_AUTO
  2. SEMI_AUTO
  3. CUSTOMIZED

Cluster events can be one of the following:

  • Nodes start and/or stop
  • Nodes experience soft and/or hard failures
  • New nodes are added/removed


[1] http://helix.apache.org/Concepts.html

0

Add a comment

We used have  Singleton Design Pattern in our applications whenever it is needed. As we know that in singleton design pattern we can create only one instance and can access in the whole application. But in some cases, it will break the singleton behavior.

13

Microservices can have a positive impact on your enterprise. Therefore it is worth to know that, how to handle Microservice Architecture (MSA) and some Design Patterns for Microservices. General goals or principles for a microservice architecture.

12

Last few years has been a great year for API Gateways and API companies. APIs (Application Programming Interfaces) are allowing businesses to expand beyond their enterprise boundaries to drive revenue through new business models.

1

kubectl (Kubernetes command-line tool) is to deploy and manage applications on Kubernetes. Using kubectl, you can inspect cluster resources; create, delete, and update components.

NOTE

You must use a kubectl version that is within one minor version difference of your cluster.

2

WSO2 Enterprise Integrator is shipped with a separate message broker profile (WSO2 MB). In this Post I will be using message broker profile in EI (6.3.0).

1

When two or many applications want to exchange data, they do so by sending the data through a channel that connects the each others.

2

Microservices are going completely over the enterprise and changed the way people write software within an enterprise ecosystem.

Let build you microservices with msf4j for Auto Mobile.

The SMPP inbound endpoint allows you to consume messages from SMSC via WSO2 ESB OR EI.

1.  Start SMSC

2.  Create custom inbound end point with below parameter. (Make sure you pick correct system-id and password correct for your SMSC)

3. Create Sequence for Inbound EP.

4. Once ESB or EI start.

1

WSO2 APIM Components

WSO2 API Manager includes five main components as the Publisher, Store, Gateway, Traffic Manager and Key Manager.

API Gateway - responsible for securing, protecting, managing, and scaling API calls.

There is REST Back-End end-point in Vehicle registration services as below

GET /car?name=prius HTTP/1.1

Host: localhost:8080

color: White

Company need to expose it 3rd part companies and above End-point should not change as internal services are using it.

1

Estimation for Software project development is the process of predicting the most realistic amount of effort (expressed in terms of person-hours or money) required to develop or maintain software based on incomplete, uncertain and noisy input.

1

In this post give some basic on JAVA Stream API which is added in Java 8. It works very well in conjunction with lambda expressions. Pipeline of stream operations can manipulate data by performing operations like search, filter, count, sort, etc.

The Lifecycle Management(LCM) plays a major role in SOA Governance. WSO2 Governance Registry Lifecycle Management supports access control at multiple levels in lifecycle state.

1.

1

Enterprise Data Integration is a broad term used in the integration landscape to connect multiple Enterprise  applications and hardware systems within an organization. All these enterprise data integration lead to achieve to remove the complexity by simplifying data management as a whole.

Post is very basic one, Since Talend is all about data integration. Finding a BigDecimal [1] in such data set is very common.

BigDecimal VS Doubles

A BigDecimal is an exact way of representing numbers. A Double has a certain precision.

Vehicles registration services using REST services on government TAX department system. That REST services give the TAX information for the Vehicle.

{"Tax": {"Amount": 58963}}

Vehicles registration Depart planning to extend the service and expose as below.

1

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. Today some systems may store data in a denormalized form and data integration tools able handle those.

There is few thing that make my work enjoyable with WSO2 ESB as it provides support for JavaScript Object Notation (JSON) payloads in messages. It is not very new feature and it old feature.

Working on an Alienvault IDS system or OSSIM you can come across over huge amount of alarms are created will system migrations.

If you’re familiar with SEIM tools or OSSEC, then you know syscheck. Syscheck is the integrity checking daemon within OSSEC. It’s purpose is simple, identify and report on changes within the system files.

Triggering action over the event occurrence in OSSIM is going to explain in this article.

There is agent in the system with IP, 192.168.80.22. Email is to be send to server admins whenever this agent disconnect and reconnect to SEIM server.

We need to have extra user data field on our security event. We need to know

event occurred time Host Server IP Editing particular event on ‘/etc/ossim/agent/plugins/ossec-single-line.cfg’. We can achieve it. We are interest on Web group and ID 0030. We added below line as our need.

Pre request

Test OSSEC new log from ‘ossec-logtest’

Here is the custom created rules.

In here I am using well known decoder in OSSEC if you need new OSSEC decoder you can write new decoder also [1]. Add new file to  rules directory in OSSEC.

Creating new OSSEC rule set

$ vi var/ossec/rules/custom_access_rules.xml

In here I am interest to monitor web user behavior model.

1

Introductions

In OSSEC, the rules are classified in multiple levels from the lowest (00) to the maximum level 16. But some levels are not used right now and below explain level details.

A brute-force attack consists of an attacker trying many passwords or passphrases with the hope of eventually guessing correctly. The attacker systematically checks all possible passwords and passphrases until the correct one is found.

Unfortunately Windows does not support Fdisk anymore. But there is another good command line tool to solve this problem. DiskPart in windows is useful format unallocated spaces in USB pen.

1. Enter ‘diskpart’ in cmd

Then disk part will start

2. List down storage in PC by

list disk

3.

The Linux kernel in Ubuntu provides a packet filtering system called netfilter, and the traditional interface for manipulating netfilter are the iptables suite of commands. The Uncomplicated Firewall (ufw) is a frontend for iptables and is particularly well-suited for host-based firewalls.

Count line when words has been matched

$ grep -c 'word' /path/to/file

Pass the -n option to precede each line of output with the number of the line in the text file

$ grep -n 'root' /etc/passwd

Ignore word case

$ grep -i 'word' /path/to/file

Use grep recursively under each directory

$ grep -r

Each application contains it's own log record format.

Access log moves to sensor / data source then I mapping to event id with considering the rules in ossim.

Data sources can be found in “ossim ->configuration –> threat_intelligence –> data_source” and search for source as below. Pick “AlienVault HIDS-accesslog” and it reads the access log.

It provides the SSH authentication to the host you want to access. For Cisco devices (PIX, routers, etc), you need to provide an additional parameter for the enable password. The same thing applies if you want to add support for “su”, it must be the additional parameter.

1. Log into AlienVault USM.

1. Download the image file of OSSIM

2. Make bootable pen with OSSIM ISO file

3. Boot drive

Make sure you have internet connection

4. Select OSSIM server to install

5. Just follow the the wizard

6. Add the net work details correctly with unique new IP for OSSIM server.

7.

Finding the logs in my server. I generally use lsof to list what is my server.

lsof | grep log

I check which log are reading by OSSEC

Check cat /var/ossec/etc/ossec.conf  |grep "<location>/"

Add new access log to OSSCE.

I am
I am
Archives
Total Pageviews
Total Pageviews
2 0 5 8 0 4 3
Categories
Categories
Loading