News

MONQ 5.0: log analysis, a superfast connection of any data source and the ability to use the system for security events

The MONQ Digital Lab team released a new version of the platform.

MONQ 5.0 now solves four main IT user cases: in addition to UX monitoring, hybrid monitoring, and process automation, the platform has log analysis capabilities. Also, now it's possible to connect immediately any data sources to the platform.

The 5th version seriously develops the possibilities of MONQ and permits to process and store any data at the input, including unstructured logs. The platform can now be used not only for events of IT monitoring systems but for any events (for example, for security events). Advanced capabilities of synthetic triggers allow combining events from infrastructure monitoring, IT security systems, and synthetic tests into one problem. Thus, a SIEM solution can be built on the MONQ platform.

Version 5.0 has fundamentally new functionality.

1. MONQ Collector. It is a new component of the platform. It replaced the connectors to monitoring systems. Its important advantage is the ability to process raw data and turn it into JSON. Any engineer of the operation department can now connect a new monitoring system or any other data source to the platform input. It hugely simplifies implementation and costs while connecting new sources.

MONQ Collector consists of the next function blocks:

  • The receiver of data streams. This is the most lightweight program - an HTTP listening point. The program wraps the basic information in a model (_id, _aggregatedAt, _connector, _sourceType, _source) and sends the message to the preprocessor for processing. The program also validates the connector key, as well as validates the input model depending on the type of input data (if the format is JSON, then the validity of JSON as a whole; if it is XML, then the validity of XML as a whole (without a schema).

  • Event preprocessor. The preprocessor starts the input message handler if it exists.

  • A handler is able to:

a) perform parsing and turning text into JSON;

b) perform processing of batch events and select single elements from these events (this is parsing: for example, events from Prometheus come in a semi-batch format);

c) perform the transformation of the input model. For example, to add a calculated field or change the field type in the model;

d) add customized tags for events.

  • The analyzer of the message database schema. The analyzer generates a model for the database, according to the schema recorded in the connector, creates this schema based on the JSON model, or adds the necessary fields. It performs data validation according to the scheme. If it finds inconsistencies in the types of fields, it indicates them in the logs.

2. A new screen for working with primary events. The screen allows users to analyze and work with received logs. In the current version, the user will be able to work with the primary events of all threads available to his workgroup. It's possible to use different tools provided for this:

  • table of primary events;

  • visualization of the number of events for the period;

  • detailed view of events;

  • auto-update events using the WebSocket protocol;

  • search strings - Lucene’s full-text search syntax is used (known by everyone familiar with Elastic Search).


Primary Data Screen

Primary Data Screen


Primary Data Screen



Primary Data Screen

Primary Data Screen


3. New functionality in the rule and action module. Now it's possible to configure not only the actions to open/close the problem but also during its confirmation.

Many events entering the system relate to existing problems and they are “confirmed”. Users would like to receive messages about it. Now a user can specify in the settings the option of performing the operation to confirm the problem. This option is available for all operations, with the exception of “Closing an Incident in HPSM.” Confirming the status of a synthetic trigger begins a check of the rules for problems already opened for this synthetic trigger. In case of successful completion of the rule, operations related to this rule marked as the “Event Confirmation” are launched. The completed “Event Confirmation” operations are displayed in the details in the “Events for the period” widget in the “Actions” tab.

4. Unification of message templates. The functionality for separating the template at the beginning and end has been removed from the message designer. The update affects all operations. Historical settings are migrated for operations that are launched both at the beginning and at the end of problems, and at the same time have different templates for the beginning and the end. Such operations are converted into two operations. One is at the beginning of the problem with the appropriate template, and the second is at the end of the problem. Activity time settings saved.

5. Adding options to run scripts. In the “Run the script” operation, launch options have been added both to confirm and to complete the problem.

6. The administrative panel has been redesigned and has new functional:

  • the possibility to write your own plugins (at the moment only for alerts, in the future this functionality will be developed to other parts of the product.

  • the libraries' management system for writing scripts. Adding your own libraries allows you to increase the possibilities of working with the Automaton module.


Work screen with the Automaton module



Work screen with the Automaton module


Work screen with the Automaton module


Learn more about MONQ during the Demo or from the presentation.

---

Some details about the platform that you may have missed.

MONQ is an AIOps platform for log analysis, UX monitoring, and automated incident management.

MONQ reduces business risks and improves financial performance that depends on the reliability of IT infrastructure and digital services. The platform increases IT productivity with AI, machine learning, and automation.

MONQ reduces the number of manually processed failure notifications, allows centralizing IT monitoring, automatically identifies and informs teams about the most important events for business. The platform automatically solves problems and has an effective notification system.
MONQ increases the efficiency of root-cause analysis and increases the speed of investigation of IT-incidents by 70-95%.

MONQ is a system that helps engineers to get rid of routine and have more time to more important things – creativity, development, and growth. Routine work remains for robots.

The MONQ Digital lab is among the winner of the Cybersecurity Challenge 2020 and Startup Village 2020.