ISSN 2415-7740 (Print) ISSN 2415-7074 (Online) Belarusian State University of Informatics and Radioelectronics Open Semantic Technologies for Intelligent Systems Research Papers Collection Founded in 2017 Issue 3 Minsk BSUIR 2019
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ISSN 2415-7740 (Print)
ISSN 2415-7074 (Online)
Belarusian State University of Informatics and Radioelectronics
Open Semantic Technologies
for Intelligent Systems
Research Papers Collection
Founded in 2017
Issue 3
Minsk
BSUIR 2019
UDC 004.822+004.89-027.31
The collection includes peer-reviewed articles approved by the Editorial board. Designed for
university professors, researchers, students, graduate students, undergraduates, as well as for
specialists of enterprises in the field of intelligent systems design.
E d i t o r i a l b o a r d :
V. V. Golenkov – Editor-in-chief,
T. A. Gavrilova, V. A. Golovko, P. S. Grabust, N. A. Guliakina, O. P.Kuznetsov,
D.V. Lande, B. M. Lobanov, А. А. Petrovskiy, V. B. Tarasov, V. F. Khoroshevsky,
A. A. Sharipbay
E d i t o r i a l a d d r e s s :
Address: Minsk, Platonova str., 39, rm 606 b
Phone: +375 (17) 293-80-92
E-mail: [email protected]
Web-site: http://proc.ostis.net
The collection is included in the Russian Science Citation Index.
© Belarusian State University of
Informatics and Radioelectronics, 2019
ISSN 2415-7740 (Print)
ISSN 2415-7074 (Online)
Учреждение образования
«Белорусский государственный университет
информатики и радиоэлектроники»
Открытые семантические технологии
проектирования интеллектуальных систем
Сборник научных трудов
Основан в 2017 году
Выпуск 3
Минск
БГУИР 2019
УДК 004.822+004.89-027.31
Сборник включает прошедшие рецензирование и утвержденные Редакционной коллегией
статьи.
Предназначен для преподавателей высших учебных заведений, научных сотрудников,
студентов, аспирантов, магистрантов, а также для специалистов предприятий в сфере
проектирования интеллектуальных систем.
Р е д а к ц и о н н а я к о л л е г и я :
В. В. Голенков – главный редактор,
Т. А. Гаврилова, В. А. Головко, П. С. Грабуст, Н. А. Гулякина, О. П. Кузнецов,
Д. В. Ландэ, Б. М. Лобанов, Г. С. Осипов, А. А. Петровский, С. В. Смирнов,
В. Б. Тарасов, В. Ф. Хорошевский, А. А. Шарипбай
А д р е с р е д а к ц и и :
Адрес: Минск, ул. Платонова,39, каб.606 б
Телефон: +375(17)293-80-92
Электронный адрес: [email protected]
Сайт: http://proc.ostis.net
Сборник включён в Российский индекс научного цитирования
© УО «Белорусский государственный
университет информатики
и радиоэлектроники», 2019
TABLE OF CONTENTS
FOREWORD 15
IN MEMORY OF ALEXANDER S. KLESHCHEV 17
THE METHODS AND THE IACPAAS PLATFORM TOOLS FOR SEMANTIC REPRESENTATION OF KNOWLEDGE AND DEVELOPMENT OF DECLARATIVE COMPONENTS FOR INTELLIGENT SYSTEMS
Valeria Gribova, Alexander Kleschev, Philip Moskalenko, Vadim Timchenko,
Leonid Fedorischev, Elena Shalfeeva
21
METHODS AND TOOLS FOR ENSURING COMPATIBILITY OF COMPUTER
SYSTEMS
Vladimir Golenkov, Natalia Guliakina, Irina Davydenko, Aleksandr Eremeev
25
PRINCIPLES OF ORGANIZATION AND AUTOMATION OF THE SEMANTIC
COMPUTER SYSTEMS DEVELOPMENT
Vladimir Golenkov, Daniil Shunkevich, Irina Davydenko, Natalia Grakova
53
PRINCIPLES OF DECISION-MAKING SYSTEMS BUILDING BASED ON THE
INTEGRATION OF NEURAL NETWORKS AND SEMANTIC MODELS
Vladimir Golovko, Aliaksandr Kroshchanka, Valerian Ivashenko, Mikhail Kovalev,
Valery Taberko, Dzmitry Ivaniuk
91
SEMANTIC ANALYSIS OF VOICE MESSAGES BASED ON A FORMALIZED
CONTEXT
Vadim Zahariev, Timofei Lyahor, Nastassia Hubarevich, Elias Azarov
103
NEURAL NETWORK BASED IMAGE UNDERSTANDING WITH
ONTOLOGICAL APPROACH
Natallia Iskra, Vitali Iskra, Marina Lukashevich
113
THE BUILDING OF THE PRODUCTION CAPACITY PLANNING SYSTEM
FOR THE AIRCRAFT FACTORY
Nadezhda Yarushkina, Anton Romanov, Aleksey Filippov, Gleb Guskov,
Maria Grigoricheva, Aleksandra Dolganovskaya
123
DYNAMIC INTEGRATED EXPERT SYSTEMS: AUTOMATED CONSTRUCTION FEATURES OF TEMPORAL KNOWLEDGE BASES WITH USING PROBLEM-ORIENTED METHODOLOGY
Galina Rybina, Ilya Sorokin, Dima Sorokin
129
HYBRID INTELLIGENT MULTIAGENT MODEL OF HETEROGENEOUS
THINKING FOR SOLVING THE PROBLEM OF RESTORING THE
DISTRIBUTION POWER GRID AFTER FAILURES
Alexander Kolesnikov, Sergey Listopad
133
5
VISUAL EVENT-SITUATIONAL APPROACH FOR INFORMATION
PREPARATION OF DECISIONS AND OPERATIONAL TECHNOLOGICAL
MANAGEMENT OF COMPLEX DYNAMIC OBJECTS
Alexander Kolesnikov
139
INFORMATION STRUCTURES IN THE FRAMEWORK OF INFORMATION
WARFARE – ONTOLOGY APPROACH
Peter Grabusts
145
APPROACH TO DETERMINING THE NUMBER OF CLUSTERS IN A DATA SET
Ivan Ishchenko, Larysa Globa, Yurii Buhaienko, Andrii Liashenko
151
APPROACH TO PREDICTION OF MOBILE OPERATORS SUBSCRIBERS CHURN
Larysa Globa, Anastasiia Moroz, Andrii Baria
155
ONTOLOGICAL APPROACH TO ANALYSIS OF BIG DATA METADATA
Julia Rogushina, Anatoly Gladun 161
ONTOLOGICAL APPROACH TO THE AUTOMATED DESIGN OF COMPLEX SYSTEMS USING AIRCRAFT AS AN EXAMPLE
Anastasiya Malochkina, Nikolai Borgest
165
COGNITIVE MAP AS REPRESENTATION OF KNOWLEDGE STRUCTURE
Svetlana Zbrishchak 169
IMPLEMENTATION OF AN ADAPTIVE MODEL OF INPUT AND EDITING INFORMATION BASED ON XSLT TRANSFORMATIONS FOR HETEROGENEOUS DATA
Aigul Mukhitova, Oleg L. Zhizhimov
173
A MODEL-DRIVEN DEVELOPMENT APPROACH FOR CASE BASES
ENGINEERING
Nikita O. Dorodnykh, Alexander Yu. Yurin
179
KNOWLEDGE MANAGEMENT SYSTEM AND DIGITAL
TRANSFORMATION OF COMPANY
Sergey Gorshkov, Roman Shebalov
183
THE NEXT STAGE OF INDUSTRY 4.0: FROM COGNITIVE TO COLLABORATIVE AND UNDERSTANDING AGENTS
Valery B. Tarassov
187
6
APPLICATION OF FUZZY MODELS OF EVOLUTIONARY DEVELOPMENT
IN OPTIMAL CONTROL OF THE SYSTEM OF PLANNED PREVENTATIVE
MAINTENANCE AND REPAIR OF EQUIPMENT FOR MULTISTAGE
PRODUCTION
Boris V. Paliukh, Alexander N. Vetrov, Irina A. Egereva, Irina I. Emelyanova
197
IMPLEMENTATION OF INTELLIGENT FORECASTING SUBSYSTEM OF
REAL-TIME
Alexander Eremeev, Alexander Kozhukhov, Natalia Guliakina
201
MULTI-CRITERIA EVALUATION OF MANAGEMENT DECISIONS IN THE
INTELLECTUAL SYSTEM OF TRANSPORTATION MANAGEMENT
Aleksandr Erofeev
205
METHODS AND TECHNOLOGIES FOR ASSESSING THE IMPACT OF ENERGY ON THE GEOECOLOGY OF A REGION (USING THE EXAMPLES OF THE BAIKAL REGION (RUSSIA) AND BELARUS)
Liudmila Massel, Alexey Massel, Tatyana Zorina
209
DESIGN PRINCIPLES OF INTEGRATED INFORMATION SERVICES FOR
BATCH MANUFACTURING ENTERPRISE EMPLOYEES
Valery Taberko, Dzmitry Ivaniuk, Valery Kasyanik, Vladimir Golovko, Kirill
Rusetski, Daniil Shunkevich, Natalia Grakova
215
EXAMPLES OF THE USE OF ARTIFICIAL NEURAL NETWORKS IN THE
ANALYSIS OF GEODATA
Valery B. Taranchuk
225
SEMANTIC TECHNOLOGY OF INTELLECTUAL GEOINFORMATION
SYSTEMS DEVELOPMENT
Sergei Samodumkin
231
ANALYSIS OF SEMANTIC PROBABILISTIC INFERENCE CONTROL
METHOD IN MULTIAGENT FORAGING TASK
Vitaly Vorobiev, Maksim Rovbo
237
ON ONTOLOGICAL MODELING OF MEASUREMENTS IN A COMPLEX
MONITORING SYSTEM OF TECHNICAL OBJECT
Maria Koroleva, Georgy Burdo
243
ATTRIBUTES, SCALES AND MEASURES FOR KNOWLEDGE
REPRESENTATION AND PROCESSING MODELS
Valerian Ivashenko
247
7
CONTROL OF A TECHNOLOGICAL CYCLE OF PRODUCTION PROCESS
BASED ON A NEURO-CONTROLLER MODEL
Viktor Smorodin, Vladislav Prokhorenko
251
POWER CONSUMPTION FOR AUTONOMOUS WIRELESS SENSOR
NETWORK NODES
Yelena Chaiko, Yelizaveta Vitulyova, Alexandr Solochshenko
257
SENSOR LOCATION PROBLEM’S SOFTWARE OPTIMIZATION
Andrei Pilipchuk, Ludmila Pilipchuk, Eugene Polyachok 261
ALGORITHM FOR FAST IMAGE COMPRESSION ON HETEROGENEOUS
COMPUTING DEVICES
Vadim V. Matskevich, Viktor V. Krasnoproshin
265
OPTIMIZING LOCAL FEATURE DESCRIPTION AND MATCHING FOR
REALTIME VIDEO SEQUENCE OBJECT DETECTION
Katsiaryna Halavataya, Vasili Sadov
269
EFFECTIVE ALGORITHM FOR OBJECT DETECTION IN THE VIDEO
STREAM FOR ARM ARCHITECTURES
Kanstantsin Kurachka, Ihar Nestsiarenia
273
DEVELOPMENT OF NEURAL NETWORK-BASED CONSULTANT
RECOGNITION METHOD FOR DETERMINING POSTURE AND BEHAVIOR
Vladimir Rozaliev, Alexey Alekseev, Andrey Ulyev, Yulia Orlova, Alexey
Petrovsky,
Alla Zaboleeva-Zotova
277
GRAPH OF TAPAZ-2 SEMANTIC CLASSIFIER
Aliaksandr Hardzei, Anna Udovichenko 281
DEVELOPMENT OF UNIVERSAL DETECTION METHODS FOR IDENTIFYING CHRONOLOGICAL OR PSEUDO-CHRONOLOGICAL ORDER OF OCCURRENCE OF TERMS IN A GIVEN SUBJECT AREA
Ekaterina Filimonova, Sergey Soloviev, Irina Polyakova
285
INFORMATION RETRIEVAL AND MACHINE TRANSLATION IN SOLVING
THE TASK OF AUTOMATIC RECOGNITION OF ADOPTED FRAGMENTS OF
TEXT DOCUMENTS
Yury Krapivin
289
RECOGNITION OF SARCASTIC SENTENCES IN THE TASK OF SENTIMENT
ANALYSIS
Alexey Dolbin, Vladimir Rozaliev, Yulia Orlova, Sergey Fomenkov
293
8
SOFTWARE MODEL OF ANALYSIS AND VISUALIZATION OF
EMOTIONAL INTONATION OF THE SPOKEN PHRASES
Boris Lobanov, Vladimir Zhitko
297
LINGUAACOUSTIC RESOURCES FOR BELARUSIAN SPEECH
SYNTHESIZERS
Evgeniya Zianouka
301
GENETIC ALGORITHM OF OPTIMIZING THE SIZE, STAFF AND NUMBER
OF PROFESSIONAL TEAMS OF PROGRAMMERS
Anatoly Prihozhy, Arseni Zhdanouski
305
TECHNOLOGIES OF INTELLIGENCE MULTIAGENT INFORMATION
PROCECCING WITH BLOCKCHAIN FOR MANAGEMENT
Vishniakou U.A., Shaya B. H., Al-Masri A. H., Al-Haji S. K.
311
EFFICIENCY OF INTELLECTUAL SYSTEM OF SECURE ACCESS IN A
PHASED APPLICATION OF MEANS OF PROTECTION CONSIDERING THE
INTERSECTION OF THE SETS OF THREAT DETECTION
Vladimir S. Kolomoitcev, Vladimir A. Bogatyrev, Vladimir I. Polyakov
315
METHOD OF DEVELOPMENT OF INFORMATION SECURITY EXPERT SYSTEM
Marzhan Tynarbay
321
CHOICE OF LIVER FAILURE TREATMENT USING SET-THEORETIC
MODELS
Nikolay A. Blagosklonov and Boris A. Kobrinskii
325
FUZZY LOGIC INFERENCE RULESET AUGMENTATION WITH SAMPLE
DATA IN MEDICAL DECISION-MAKING SYSTEMS
Alexander Kurochkin, Vasili Sadov
329
DO INTELLECTUAL SYSTEMS NEED EMOTIONS?
Maksim Davydov, Anatoly Osipov, Sergei Kilin, Vladimir Kulchitsky 333
ALGORITHM OF GENERATION FINITE ELEMENT MESH FOR THE
SYSTEM «VERTEBRAE – INTERVERTEBRAL DISK – VERTEBRAE» BASED
ON THE STL MODEL
Konstantin Kurochka, Kosntantin Panarin, Ekaterina Karabchikova
337
AUTHOR INDEX 341
9
СОДЕРЖАНИЕ
ПРЕДИСЛОВИЕ 15
ПАМЯТИ КЛЕЩЕВА АЛЕКСАНДРА СЕРГЕЕВИЧА 17
МЕТОДЫ И СРЕДСТВА ПЛАТФОРМЫ IACPAAS ДЛЯ СЕМАНТИЧЕСКОГО ПРЕДСТАВЛЕНИЯ ЗНАНИЙ И РАЗРАБОТКИ ДЕКЛАРАТИВНЫХ КОМПОНЕНТОВ ИНТЕЛЛЕКТУАЛЬНЫХ СИСТЕМ
Грибова В. В., Клещев А. С., Москаленко Ф. М., Тимченко В. А., Федорищев Л. А., Шалфеева Е. А.
21
МЕТОДЫ И СРЕДСТВА ОБЕСПЕЧЕНИЯ СОВМЕСТИМОСТИ КОМПЬЮТЕРНЫХ СИСТЕМ
Голенков В. В., Гулякина Н. А., Давыденко И. Т., Еремеев А. П.
25
ПРИНЦИПЫ ОРГАНИЗАЦИИ И АВТОМАТИЗАЦИИ ПРОЦЕССА РАЗРАБОТКИ СЕМАНТИЧЕСКИХ КОМПЬЮТЕРНЫХ СИСТЕМ
Голенков В. В., Шункевич Д. В., Давыденко И. Т., Гракова Н. В.
53
ПРИНЦИПЫ ПОСТРОЕНИЯ СИСТЕМ ПРИНЯТИЯ РЕШЕНИЙ НА ОСНОВЕ ИНТЕГРАЦИИ НЕЙРОСЕТЕВЫХ И СЕМАНТИЧЕСКИХ МОДЕЛЕЙ
Головко В. А., Крощенко А. А., Таберко В. В., Иванюк Д. С., Ивашенко В. П., Ковалев М. В.
91
СЕМАНТИЧЕСКИЙ АНАЛИЗ РЕЧЕВЫХ СООБЩЕНИЙ НА ОСНОВЕ ФОРМАЛИЗОВАННОГО КОНТЕКСТА
Захарьев В. А., Ляхор Т. В., Губаревич А. В., Азаров И. С.
103
НЕЙРОСЕТОВОЕ РАСПОЗНАВАНИЕ ИЗОБРАЖЕНИЙ С ИСПОЛЬЗОВАНИЕМ ОНТОЛОГИЧЕСКОГО ПОДХОДА
Искра Н. А., Искра В. В., Лукашевич М. М.
113
ПОСТРОЕНИЕ СИСТЕМЫ БАЛАНСА ПРОИЗВОДСТВЕННЫХ МОЩНОСТЕЙ АВИАЦИОННОГО ЗАВОДА
Ярушкина Н. Г., Романов А. А., Филиппов А. А., Гуськов Г. Ю., Григоричева М. С., Долгановская А. Ю.
123
ДИНАМИЧЕСКИЕ ИНТЕГРИРОВАННЫЕ ЭКСПЕРТНЫЕ СИСТЕМЫ ОСОБЕННОСТИ АВТОМАТИЗИРОВАННОГО ПОСТРОЕНИЯ ТЕМПОРАЛЬНЫХ БАЗ ЗНАНИЙ НА ОСНОВЕ ЗАДАЧНО-ОРИЕНТИРОВАННОЙ МЕТОДОЛОГИИ
Рыбина Г. В., Сорокин И. А., Сорокин Д. О.
129
ГИБРИДНАЯ ИНТЕЛЛЕКТУАЛЬНАЯ МНОГОАГЕНТНАЯ МОДЕЛЬ ГЕТЕРОГЕННОГО МЫШЛЕНИЯ ДЛЯ РЕШЕНИЯ ЗАДАЧИ ВОССТАНОВЛЕНИЯ РАСПРЕДЕЛИТЕЛЬНОЙ ЭЛЕКТРОСЕТИ ПОСЛЕ АВАРИЙ
Колесников А. В., Листопад С. В.
133
10
ВИЗУАЛЬНЫЙ СОБЫТИЙНО-СИТУАЦИОННЫЙ ПОДХОД ИНФОРМАЦИОННОЙ ПОДГОТОВКИ РЕШЕНИЙ И ОПЕРАТИВНО-ТЕХНОЛОГИЧЕСКОГО УПРАВЛЕНИЯ СЛОЖНЫМИ ДИНАМИЧЕСКИМИ ОБЪЕКТАМИ
Колесников А. В.
139
ИНФОРМАЦИОННЫЕ СТРУКТУРЫ В КОНТЕКСТЕ ИНФОРМАЦИОННЫХ ВОЙН – ИСПОЛЬЗОВАНИЕ ОНТОЛОГИЙ
Грабуст П. С.
145
ПОДХОД К ОПРЕДЕЛЕНИЮ КОЛИЧЕСТВА КЛАСТЕРОВ В НАБОРЕ ДАННЫХ
Ищенко И. А., Глоба Л. С., Бугаенко Ю. М., Ляшенко А. В.
151
ПРЕДСКАЗАНИЕ ОТТОКА АБОНЕНТОВ ОТ ОПЕРАТОРОВ МОБИЛЬНОЙ СВЯЗИ
Баря А. Д., Глоба Л. С., Мороз А. М.
155
ИСПОЛЬЗОВАНИЕ ОНТОЛОГИЙ ДЛЯ АНАЛИЗА МЕТАДАННЫХ BIG DATA
Рогушина Ю. В., Гладун А. Я.
161
ОНТОЛОГИЧЕСКИЙ ПОДХОД К ПРОЕКТИРОВАНИЮ СЛОЖНЫХ АВТОМАТИЗИРОВАННЫХ СИСТЕМ НА ПРИМЕРЕ САМОЛЕТА
Малочкина А. В., Боргест Н. М.
165
КОГНИТИВНАЯ КАРТА КАК РЕПРЕЗЕНТАЦИЯ СТРУКТУР ЗНАНИЙ
Збрищак С. Г. 169
РЕАЛИЗАЦИЯ АДАПТИВНОЙ МОДЕЛИ ВВОДА И РЕДАКТИРОВАНИЯ ИНФОРМАЦИИ НА ОСНОВЕ XSLT-ПРЕОБРАЗОВАНИЙ ДЛЯ РАЗНОРОДНЫХ ДАННЫХ
Мухитова А., Жижимов О. Л.
173
РАЗРАБОТКА ПРЕЦЕДЕНТНЫХ БАЗ ЗНАНИЙ С ИСПОЛЬЗОВАНИЕМ MDE-ПОДХОДА
Дородных Н. О., Юрин А. Ю.
179
СИСТЕМА УПРАВЛЕНИЯ ЗНАНИЯМИ И ЦИФРОВАЯ ТРАНСФОРМАЦИЯ КОМПАНИИ
Горшков С. В., Шебалов Р. Ю.
183
CЛЕДУЮЩАЯ СТАДИЯ ИНДУСТРИИ 4.0: ОТ КОГНИТИВНЫХ К КОЛЛАБОРАТИВНЫМ И «ПОНИМАЮЩИМ» АГЕНТАМ
Тарасов В. Б.
187
11
ОПТИМАЛЬНОЕ УПРАВЛЕНИЕ СИСТЕМОЙ ПЛАНОВО-ПРЕДУПРЕДИТЕЛЬНОГО РЕМОНТА ОБОРУДОВАНИЯ МНОГОСТАДИЙНОГО ПРОИЗВОДСТВА С ПРИМЕНЕНИЕМ НЕЧЕТКИХ МОДЕЛЕЙ ЭВОЛЮЦИОННОГО РАЗВИТИЯ
Палюх Б. В., Ветров А. Н., Егерева И. А., Емельянова И. И.
197
РЕАЛИЗАЦИЯ ИНТЕЛЛЕКТУАЛЬНОЙ ПОДСИСТЕМЫ ПРОГНОЗИРОВАНИЯ РЕАЛЬНОГО ВРЕМЕНИ
Еремеев А. П., Кожухов А. А., Гулякина Н. А.
201
МНОГОКРИТЕРИАЛЬНАЯ ОЦЕНКА УПРАВЛЕНЧЕСКИХ РЕШЕНИЙ В ИНТЕЛЛЕКТУАЛЬНОЙ СИСТЕМЕ УПРАВЛЕНИЯ ПЕРЕВОЗОЧНЫМ ПРОЦЕССОМ
Ерофеев А. А.
205
МЕТОДЫ И ТЕХНОЛОГИИ ОЦЕНКИ ВОЗДЕЙСТВИЯ ЭНЕРГЕТИКИ НА ГЕОЭКОЛОГИЮ РЕГИОНА (НА ПРИМЕРЕ БАЙКАЛЬСКОГО РЕГИОНА (РОССИЯ) И БЕЛАРУСИ)
Массель Л. В., Массель А. Г., Зорина Т. Г.
209
ПРИНЦИПЫ ПОСТРОЕНИЯ СИСТЕМЫ КОМПЛЕКСНОГО ИНФОРМАЦИОННОГО ОБСЛУЖИВАНИЯ СОТРУДНИКОВ ПРЕДПРИЯТИЯ РЕЦЕПТУРНОГО ПРОИЗВОДСТВА
Таберко В. В., Иванюк Д. С., Касьяник В. В., Головко В. А., Русецкий К. В., Шункевич Д. В., Гракова Н. В.
215
ПРИМЕРЫ ИСПОЛЬЗОВАНИЯ НЕЙРОННЫХ СЕТЕЙ В АНАЛИЗЕ ГЕОДАННЫХ
Таранчук В. Б.
225
СЕМАНТИЧЕСКАЯ ТЕХНОЛОГИЯ ПРОЕКТИРОВАНИЯ ИНТЕЛЛЕКТУАЛЬНЫХ ГЕОИНФОРМАЦИОННЫХ СИСТЕМ
Самодумкин С. А.
231
АНАЛИЗ МЕТОДА УПРАВЛЕНИЯ НА ОСНОВЕ СЕМАНТИЧЕСКОГО ВЕРОЯТНОСТНОГО ВЫВОДА В МНОГОАГЕНТНОЙ ЗАДАЧЕ ФУРАЖИРОВКИ
Воробьев В. В., Ровбо М. А.
237
ОБ ОНТОЛОГИЧЕСКОМ МОДЕЛИРОВАНИИ ИЗМЕРЕНИЙ В КОМПЛЕКСНОЙ СИСТЕМЕ МОНИТОРИНГА ТЕХНИЧЕСКОГО ОБЪЕКТА
Королева М. Н., Бурдо Г. Б.
243
ПРИЗНАКИ, ШКАЛЫ И МЕРЫ ДЛЯ МОДЕЛЕЙ ПРЕДСТАВЛЕНИЯ И ОБРАБОТКИ ЗНАНИЙ
Ивашенко В. П.
247
УПРАВЛЕНИЕ ТЕХНОЛОГИЧЕСКИМ ЦИКЛОМ ПРОИЗВОДСТВА НА ОСНОВЕ МОДЕЛИ НЕЙРОКОНТРОЛЛЕРА
Смородин В. С., Прохоренко В. А.
251
12
ПОТРЕБЛЕНИЕ ЭНЕРГИИ АВТОНОМНЫМИ УЗЛАМИ В БЕСПРОВОДНЫХ СЕНСОРНЫХ СЕТЯХ
Чайко Е. В., Витулёва Е. С., Солощенко А. В.
257
ОПТИМИЗАЦИЯ ПРОГРАММНОГО ОБЕСПЕЧЕНИЯ ПРОБЛЕМЫ РАСПОЛОЖЕНИЯ СЕНСОРОВ
Пилипчук А. С., Пилипчук Л. А., Полячок Е. Н.
261
АЛГОРИТМ БЫСТРОГО СЖАТИЯ ИЗОБРАЖЕНИЙ НА ГЕТЕРОГЕННЫХ ВЫЧИСЛИТЕЛЬНЫХ УСТРОЙСТВАХ
Мацкевич В. В., Краснопрошин В. В.
265
ОПТИМИЗАЦИЯ АЛГОРИТМОВ ОПИСАНИЯ И СРАВНЕНИЯ ЛОКАЛЬНЫХ ПРИЗНАКОВ ИЗОБРАЖЕНИЙ ПРИ ДЕТЕКТИРОВАНИИ ОБЪЕКТОВ НА ВИДЕОПОСЛЕДОВАТЕЛЬНОСТЯХ В РЕАЛЬНОМ ВРЕМЕНИ
Головатая Е. А., Садов В. С.
269
ЭФФЕКТИВНЫЙ АЛГОРИТМ ДЕТЕКТИРОВАНИЯ ОБЪЕКТОВ НА ВИДЕОПОТОКЕ АДАПТИРОВАННЫЙ ДЛЯ ARM АРХИТЕКТУРЫ
Курочка К. С., Нестереня И. Г.
273
РАЗРАБОТКА МЕТОДА РАСПОЗНАВАНИЯ ПРОДАВЦОВ-КОНСУЛЬТАНТОВ НА ОСНОВЕ НЕЙРОСЕТИ ДЛЯ ОПРЕДЕЛЕНИЯ ПОЗЫ И ПОВЕДЕНИЯ
Розалиев В. Л., Алексеев А. В., Ульев А. Д., Орлова Ю. А., Петровский А. Б., Заболеева-Зотова А. В.
277
ГРАФ СЕМАНТИЧЕСКОГО КЛАССИФИКАТОРА ТАПАЗ-2
Гордей А. Н., Удовиченко А. М. 281
РАЗРАБОТКА УНИВЕРСАЛЬНЫХ МЕТОДОВ ВЫЯВЛЕНИЯ ХРОНОЛОГИЧЕСКОГО ИЛИ ПСЕВДОХРОНОЛОГИЧЕСКОГО ПОРЯДКА ВОЗНИКНОВЕНИЯ ТЕРМИНОВ В ЗАДАННОЙ ПРЕДМЕТНОЙ ОБЛАСТИ
Филимонова Е. А., Соловьев С. Ю., Полякова И. Н.
285
ИНФОРМАЦИОННЫЙ ПОИСК И МАШИННЫЙ ПЕРЕВОД В РЕШЕНИИ ЗАДАЧИ АВТОМАТИЧЕСКОГО РАСПОЗНАВАНИЯ ЗАИМСТВОВАННЫХ ФРАГМЕНТОВ ТЕКСТОВЫХ ДОКУМЕНТОВ
Крапивин Ю. Б.
289
РАСПОЗНАВАНИЕ ПРЕДЛОЖЕНИЙ СОДЕРЖАЩИХ САРКАЗМ В ЗАДАЧЕ АНАЛИЗА ТОНАЛЬНОСТИ
Долбин А. В., Розалиев В. Л., Орлова Ю. А., Фоменков С. А.
293
ПРОГРАММНАЯ МОДЕЛЬ АНАЛИЗА И ВИЗУАЛИЗАЦИИ ЭМОЦИОНАЛЬНОЙ ИНТОНАЦИИ В УСТНОЙ РЕЧИ
Лобанов Б. М., Житко В. А.
297
13
ЛИНГВОАККУСТИЧЕСКИЕ РЕСУРСЫ ДЛЯ БЕЛОРУССКОЯЗЫЧНЫХ СИСТЕМ СИНТЕЗА РЕЧИ
Зеновко Е. С.
301
ГЕНЕТИЧЕСКИЙ АЛГОРИТМ ОПТИМИЗАЦИИ ЧИСЛЕННОСТИ, ПЕРСОНАЛА И КОЛИЧЕСТВА ПРОФЕССИОНАЛЬНЫХ КОМАНД ПРОГРАММИСТОВ
Прихожий А. А., Ждановский А. М.
305
ИСПОЛЬЗОВАНИЕ ТЕХНОЛОГИИ ИНТЕЛЛЕКТУАЛЬНОЙ МНОГОАГЕНТНОЙ ОБРАБОТКИ ИНФОРМАЦИИ С БЛОКЧЕЙН ДЛЯ СИСТЕМ УПРАВЛЕНИЯ
Вишняков В. А., Шайя Б. Х., Эль Масри А. Х., Эль Хаджи С. К.
311
ЭФФЕКТИВНОСТЬ ИНТЕЛЕКТУАЛЬНОЙ СИСТЕМЫ БЕЗОПАСНОГО ДОСТУПА ПРИ ПОСЛЕДОВАТЕЛЬНОМ ПРИМЕНЕНИИ СРЕДСТВ ЗАЩИТЫ С УЧЕТОМ ПЕРЕСЕКАЕМОСТИ МНОЖЕСТВ ОБНАРУЖЕНИЯ УГРОЗ
Коломойцев В. С., Богатырев В. А., Поляков В. И.
315
МЕТОД РАЗРАБОТКИ ЭКСПЕРТНОЙ СИСТЕМЫ ИНФОРМАЦИОННОЙ БЕЗОПАСНОСТИ
Тынарбай М.
321
ВЫБОР АНАЛОГИЧНЫХ МЕТОДОВ ЛЕЧЕНИЯ ПЕЧЕНОЧНОЙ НЕДОСТАТОЧНОСТИ ПРИ ИСПОЛЬЗОВАНИИ ТЕОРЕТИКО-МНОЖЕСТВЕННЫХ МОДЕЛЕЙ
Благосклонов Н. А., Кобринский Б. А.
325
УТОЧНЕНИЕ НАБОРА ПРАВИЛ СИСТЕМЫ НЕЧЕТКОГО ВЫВОДА С ИСПОЛЬЗОВАНИЕМ ИСТОРИЧЕСКИХ ДАННЫХ В МЕДИЦИНСКИХ СИСТЕМАХ ПОДДЕРЖКИ ПРИНЯТИЯ РЕШЕНИЙ
Курочкин А. В., Садов В. С.
329
НУЖНЫ ЛИ ИНТЕЛЛЕКТУАЛЬНЫМ СИСТЕМАМ ЭМОЦИИ?
Давыдов М. В., Осипов А. Н., Килин С. Я, Кульчицкий В. А. 333
АЛГОРИТМ ГЕНЕРАЦИИ КОНЕЧНОЭЛЕМЕНТНОЙ СЕТКИ ДЛЯ СИСТЕМЫ «ПОЗВОНОК – МЕЖПОЗВОНОЧНЫЙ ДИСК – ПОЗВОНОК» НА ОСНОВЕ STL МОДЕЛИ
Курочка К. С., Панарин К. А., Карабчикова Е. А.
337
АВТОРСКИЙ УКАЗАТЕЛЬ 341
14
ПРЕДИСЛОВИЕ
Сборник научных трудов «Открытые семантические технологии проектирования
интеллектуальных систем» посвящен вопросам разработки гибких и совместимых технологий,
обеспечивающих быстрое и качественное построение интеллектуальных систем различного
назначения.
В сборнике отражены исследования в сфере искусственного интеллекта по следующим
направлениям:
• Гибридные интеллектуальные системы;
• Интеллектуальные человеко-машинные системы;
• Робототехника;
• Компьютерное зрение;
• Нечеткие вычисления;
• Интеллектуальные агенты;
• Интеллектуальная автоматизация;
• Интеллектуальное управление;
• Большие данные;
• Управление знаниями;
• Инженерия знаний;
• Онтологическое проектирование;
• Семантические сети;
• Машинное обучение;
• Распознавание образов;
• Искусственные нейронные сети;
• Обработка текстов естественного языка;
• Обработка речи.
Первая статья в данном выпуске сборника посвящена памяти А.С. Клещева, действительного
члена Академии инженерных наук РФ, руководителя Лаборатории интеллектуальных систем
Института автоматики и процессов управления ДВО РАН, который проводил исследования в сфере
разработки интеллектуальных систем.
Основной акцент в этом выпуске сборника сделан на проблему обеспечения совместимости
интеллектуальных систем.
В общей сложности сборник содержит 50 статей. Редакция сборника благодарит всех авторов,
представивших свои статьи. Для публикации научными экспертами были отобраны лучшие из
представленных работ, многие из них были переработаны в соответствии с замечаниями
рецензентов.
Мы также благодарим экспертов за большой труд по рецензированию статей в тесном
взаимодействии с авторами, чей труд позволил повысить уровень изложения научных результатов,
а также создал платформу для дальнейших научных дискуссий.
Надеемся, что, как и прежде, сборник будет выполнять свою основную функцию –
способствовать активному сотрудничеству между бизнесом, наукой и образованием в области
искусственного интеллекта.
Главный редактор
Голенков Владимир Васильевич
15
FOREWORD
Research papers collection “Open Semantic Technology for Intelligent Systems” is devoted to the
flexible and compatible technologies development that ensure the rapid and high-quality design of
intelligent systems for various purposes.
The collection reflects research in the field of artificial intelligence in the following areas:
Hybrid Intelligent Systems;
Intelligent human-machine systems;
Robotics;
Computer Vision;
Fuzzy Computing ;
Intelligent Agents ;
Intelligent Automation;
Intelligent Control;
Big Data;
Knowledge management;
Knowledge Engineering;
Ontological design;
Semantic Networks;
Machine Learning;
Pattern Recognition;
Neural Networks;
Natural Language Processing;
Speech Processing.
The first article is dedicated to the memory of A.S. Kleshcheva, member of the Academy of
Engineering Sciences of the Russian Federation, head of the Laboratory of Intelligent Systems of the
Institute of Automation and Control Processes of the Far Eastern Branch of the Russian Academy of
Sciences, who conducted research in the field of intelligent systems development.
The main focus of this issue is on the problem of ensuring the compatibility of intelligent systems.
In total, the collection contains 50 articles. The editors are thankful for all authors who sent their
articles. Scientific experts selected for publication the best of the submitted works, many of them were
revised in accordance with the comments of reviewers.
We are grateful our scientific experts for their great job in reviewing the articles in close cooperation
with the authors. Their work allowed to raise the level of scientific results presentation, and also created a
platform for further scientific discussions.
We hope that, as before, the collection will perform its main function -- to promote active cooperation
between business, science and education in the field of artificial intelligence.
Editor-in-chief
Golenkov Vladimir
16
Памяти Клещева Александра Сергеевича
10 января 2019 года ушел из жизни Заслуженныйдеятель науки Российской Федерации, действитель-ный член Академии инженерных наук РФ, главныйнаучный сотрудникИнститута автоматики и процессовуправления ДВОРАН, д.ф.-м.н., профессорАлександрСергеевич Клещев.
КлещевАлександрСергеевич родился в 1940 г. вЛе-нинграде, в 1964 г. окончил математико-механическийфакультет Ленинградского государственного универ-ситета, в 1973 г. в Институте кибернетики АН УССР(г. Киев) защитил диссертацию на соискание ученойстепени кандидатафизико-математических наук на те-му «Реализация многоцелевых динамических языковпрограммирования», в 1990 г. в Институте прикладнойматематики им. М.В. Келдыша АН СССР (г. Москва)защитил диссертацию на соискание ученой степенидоктора физико-математических наук на тему «Реа-лизация экспертных систем на основе декларативныхмоделей представления знаний».
Свою профессиональную деятельность АлександрСергеевич начал в ВЦ Ленинградского нейрохирурги-ческого института им. А.Л. Поленова (1963 - 1968 гг.)в должности инженера-программиста. Им совместнос В.Л. Темовым был разработан многоцелевой ди-намический язык ИНФ, а также транслятор с этогоязыка для ЭВМ «Днепр-21». С этими результатамиони, как сформулировалС.С. Лавров в обзорной статье«Ленинградская школа программирования» «как быкометой взошли над программистским горизонтом».При активном участии Александра Сергеевича былразработан и первый в СССР компилятор языка вы-сокого уровня, внедренный в 35 вычислительных цен-трах различного профиля в 11 городах СССР. Таковынекоторые из его достижений в области системногопрограммирования.
Рис. 1. А.С. Клещев (предположительно 1974 г.)
В 1968-1974 годах он работал ведущим инженером17
ВЦ биологических институтов АН СССР при Инсти-туте физиологии им. И.П. Павлова (г. Ленинград). Вобласти биологической и медицинской кибернетикипри его участии была разработана программная систе-ма автоматической диагностики нейрохирургическихзаболеваний, построен ряд математических моделейпроцессов адаптации в центральной нервной системе,механизмов обучения и запоминания.
С 1974 года и до последнего дня своей жизниАлександр Сергеевич работал в Институте автоматикии процессов управления ДВО РАН (г. Владивосток).Более тридцати лет он возглавлял сначала отдел экс-пертных систем, затемлабораториюинтеллектуальныхсистем. На протяжении многих лет он заведовал со-зданной им в 1990 г. кафедрой программного обеспе-чения ЭВМ Дальневосточного государственного уни-верситета, был деканом факультета компьютерныхнаук.
Рис. 2. Обсуждение новой идеи. Слева направо:М.Ю.Черняховская,А.С. Клещев, В.В. Грибова
Рис. 3. Субботник в ИАПУ ДВО РАН 27 апреля 2007 г.
Рис. 4. Субботник в ИАПУ ДВО РАН 27 апреля 2007 г.
Александр Сергеевич был организатором и предсе-дателем жюри многолетнего конкурса компьютерныхпрограмм студентов, аспирантов и молодых специали-стов «Программист», который более 10 лет проводилсяна базе Дальневосточного Государственного Универ-ситета.
А.С. Клещев активно развивал искусственный ин-теллект на Дальнем Востоке, был Председателем Вла-дивостокского регионального отделения Российскойассоциации искусственного интеллекта. Им были на-чаты работы по построению математической теорииразработки экспертных систем, моделей представле-ния знаний, методов их обработки, а также моделями методам разработки специального программногообеспечения базового и прикладного уровней.
Во второй половине 1980-х годов началось исследо-вание методов трансляции систем реляционных кон-флюентных продукций, основанных на идее вывода,управляемого потоком. В конце 1980-х годов на ос-нове этих результатов была разработана переносимаяпроблемно-независимая оболочка экспертных системтрансляционного типа для вычислительных машинсерии ЕС и персональных компьютеров.
Разработка теории онтологий предметных областейи методов реализации интеллектуальных систем на их
18
основе началась в 1977 году с исследования возможно-сти представления в экспертной системе диагностикиострых заболеваний медицинских знаний в форме,более естественной для врачей, чем это позволялиделать универсальные модели представлений знаний –системы продукций и фреймов. Эти работы привели кпостроению метаонтологии медицинской диагностикиострых заболеваний, которая была представлена вмонографии М.Ю. Черняховской (Представление зна-ний в экспертных системах медицинской диагностике.Владивосток: ДВНЦ АН СССР, 1983).
Были обоснованы возможности построения теориии технологии разработки экспертных систем и об-работки знаний на основе декларативных моделей.Выполненные теоретические исследования положеныв основу разработанных под его руководством инстру-ментальных средств, в том числе генератора управля-емых онтологиями интеллектуальных редакторов баззнаний.
Под руководством Александра Сергеевича был раз-работан ряд прикладных интеллектуальных систем,среди которых первые в СССР экспертные системы винтересахМинистерства обороны: экспертная системаидентификации подводных лодок на Тихом океане,геоинформационная экспертная система реальноговремени для обнаружения ситуаций угрозы силамТихоокеанского флота и прогнозирования развитияэтих ситуаций. В рамках международного проекта сЯпонией была разработана экспертная система, улуч-шающая проекты программ для станков с ЧПУ наоснове базы know-how, собранной ведущими япон-скими фирмами в этой области. В области медициныразработана система диагностики острого живота длямашин серии ЕС. Данная экспертная система былаустановлена на большой разведывательный атомныйкорабль «Урал». И это только некоторые из при-кладных систем, разработанные при активном участииАлександра Сергеевича.
Помимо прикладных интеллектуальных систем онактивно развивал направление исследований, связан-ное с разработкой инструментальных систем для ихсоздания.
С начала 2000-х годов Александром Сергееви-чем были начаты работы по применению техно-логии облачных вычислений для разработки и ис-пользования интеллектуальных систем. В результатебыла создана компьютерная платформа «Многоце-левой банк знаний», включающая информационно-административную систему, хранилище информаци-онных ресурсов различных уровней общности, уда-ленный универсальный редактор для создания и моди-фикации информационных ресурсов, а также единыйпрограммный интерфейс для их обработки программ-ными сервисами. С использованием Многоцелевогобанка знаний были разработаны специализированные
компьютерные банки знаний для ряда предметныхобластей, таких как медицина, математика, преобра-зование программ, включающие тематические инфор-мационные ресурсы, их редакторы, а также интернет-приложения – интеллектуальные системы для решениязадач в этих предметных областях.
Полученный опыт позволил перейти к следующейверсии этого проекта, которая получила названиеIACPaaS – облачная платформа для разработки, управ-ления и удаленного использования как прикладныхинтеллектуальных облачных сервисов, так и инстру-ментария для создания, сопровождения и обеспе-чения жизнеспособности таких сервисов. В данныйинструментарий заложены механизмы расширения егофункциональных возможностей как разработчиками,так и пользователями этого инструментария.Этимеха-низмы основаны на едином декларативном представ-лении информационных и программных компонентовинструментария с возможностью автоматической ге-нерации редакторов для их формирования.
Александр Сергеевич подготовил 6 докторов и 15кандидатов наук, был членом двух диссертацион-ных советов по защитам докторских диссертаций,членом Объединенного ученого совета по физико-математическим и техническим наукам ДВО РАН,членом редколлегий ряда Российских и междуна-родных журналов, руководителем ряда российских имеждународных проектов. Им опубликовано более 360научных публикаций (в том числе двух монографий)в области искусственного интеллекта, информатики,медицинской и биологической кибернетики.
Рис. 5. Обсуждение перспектив технологий разработки интеллекту-альных систем (ИВМиМГ СО РАН, г. Новосибирск)
Александр Сергеевич тесно сотрудничал с Инсти-тутом систем информатики СО РАН, который потрадиции организует выступления известных ученыхи специалистов в области информатики, программи-рования и вычислительной техники. Он был частым и
19
Рис. 6. Лекция «Современные технологии разработки интеллекту-альных систем» (конферец-зал ИВМиМГ СО РАН, г. Новосибирск)
желанным гостем Ершовских лекций по информатике.В частности, в 2014 году он сделал доклад на тему«Современные технологии разработки интеллектуаль-ных систем».
Эрудиция и широта интересов Александра Сергее-вича были поразительными. Он родился в семье музы-кантов (мама была преподавателем по классу скрипкив консерватории) и всегда интересовался классическоймузыкой, собрал большую библиотеку записей музы-кальных произведений от разных исполнителей (более2000 сборников).
Также в центре внимания Александра Сергеевичавсегда были философия и живопись. Он написал 11томов (228 глав) об истории европейской авторскойживописи с анализом истории, политической жизни,культуры средних веков и ренессанса и их влиянияна произведения европейских художников, особенносвязанных с христианскими сюжетами. Сейчас егоистория живописи готовится к изданию.
В 2016 г. А. С. Клещев передал в дар Приморскойгосударственной картинной галерее семейную релик-вию – этюд З.Е. Серебряковой «Тата и Катя», которыйбыл подарен маме Александра Сергеевича подругойхудожницы.
Мы запомнили Александра Сергеевича как исклю-чительно интеллигентного и позитивного человека, сего лица никогда не сходила улыбка.Унего всегда быломного неординарных идей, которыми он с большимудовольствием делился. Он был добрым и открытымчеловеком, любил помогать не только сотрудникамлаборатории и института, но и всем, кто просил егоо помощи.
Александр Сергеевич был прекрасным оратором.Его выступления, доклады и лекции были очень яр-кими, эмоциональными и убедительными. Он любилжизнь и до последнего дня своей жизни был полон
Рис. 7. Александр Сергеевич со своей любимой супругой М.Ю.Черняховской
идей и планов.Память об этом замечательном человеке навсегда
сохранится в сердцах его соратников, коллег, учени-ков.
20
The methods and the IACPaaS Platform toolsfor semantic representation of knowledge and
development of declarative components forintelligent systems
Valeria Gribova, Alexander Kleschev, Philip MoskalenkoVadim Timchenko, Leonid Fedorischev, Elena Shalfeeva
IACP FEB RASVladivostok, Russian Federation
[email protected], [email protected], [email protected]@dvo.ru, [email protected], [email protected]
Abstract—The paper discusses the problem of ensuringthe viability of intelligent systems – systems with declarativeknowledge bases. Software tools for the development ofsuch systems that implement mechanisms for viabilityimprovement are considered. These mechanisms are basedon the construction of each component according to itsdeclarative model, which is specified in a unified languagefor model description.
Keywords—intelligent systems, software system mainte-nance, software system viability, development tools
I. INTRODUCTION
Ensuring the viability of software systems (SS) isone of the key problems in software engineering. Theterm viability refers to the SS sustainability (perfor-mance preservation) to changes in the environment andthe ability to evolve during the lifecycle [1], [2], [3].Viability is directly related to the SS transparency, whichis characterized by three main properties: accessibility,clarity and relevance of the information and componentsof the SS to interest groups [4].
Among the many SSs, the class of intelligent systemsis distinguished. They are systems with knowledge bases(KBS), which are actively used to solve various scientificand applied problems. Their architecture, among the tra-ditional components – databases, business logic (solver)and user interface, contains an additional component –the knowledge base. At present, one can say that KBSshave reached the phase of maturity. But the problem ofensuring their viability is acute, since the developmentteam of such systems includes knowledge engineersand domain experts in addition to programmers andinterface designers. This class of SS is characterized bya continuous improvement of knowledge bases, and an
The work is carried out with the partial financial support of theRFBR (projects nos. 19-07-00244, 18-07-01079) and by PFI “Far East”(project no 18-5-078).
occasional improvement of problem solving method andof an output explanation.
Despite the development of tools for creating of sys-tems of this class, the problem of their viability remainsurgent:
• domain experts still cannot independently (without in-termediaries like knowledge engineers and programmers)build and maintain knowledge bases;
• part of the domain knowledge is “embedded” into theproblem solver, which makes their modification moredifficult, and its structure is hard to understand;
• the UI does not adapt to the requirements of users, of theplatform, of the domain, it usually has a “firm” structurebuilt into the problem solver.
These drawbacks make it necessary to use additionalspecialized mechanisms to ensure the viability of thisclass of systems. The aim of the work is to describe newmodels and methods aimed at providing the viability ofthe KBS.
II. REVIEW
There are three main types of KBS development tools:programming languages, shells and specialized tool sys-tems. General purpose programming languages (Python,C#, Java, etc.) or specialized ones (LISP, Smalltalk, FRL,etc.) are universal development tools. In [5] it is notedthat the complexity of intelligent system developmentwith the use of programming languages is so great thatit is practically unaffordable.
Problem-independent and specialized shells greatlysimplify the creation of the KBS, however, they limitthe possibilities of their evolution: they have a pre-defined solver and an embedded UI that cannot bemodified if requirements change. Also, the disadvantagesof specialized shells include limitations on the field oftheir use, and disadvantages of the problem-independentones – their “non-transparency” primarily for domainexperts who cannot independently (without knowledge
21
engineers) form and maintain a knowledge base as part ofthe knowledge is built into the logical inference machine[5], [6].
Specialized tool systems are focused on a wide class ofKBS. Typical representatives of tool systems are: Level5Object, G2, Clips, Loops, VITAL, KEATS, OSTIS, AT-technology etc. [5], [6], [7]. They differ by the knowledgerepresentation formalisms, by the used output mecha-nisms, and by the tools for UI forming.
Looking at these tools from the point of view ofthe viability of KBS created with their use, it can benoted that the evolutionary development of tool systemsis focused on achieving this important goal in one wayor another. It is primarily reflected in the tools whichsupport the knowledge base (KB) creation, which isone of the most difficult stages of development of suchsystems, as well as in methods of coupling of KB witha problem solver.
According to [6], the most common model of knowl-edge representation remains the rule-based one. But bynow, the trend of production model systems amountreducing is obvious. Given the need for alternativeknowledge representation models, many developmenttools offer a mixed mechanism for their presentation. Forexample, LOOP and G2 use rules and object-orientedrepresentation, ART – rules, frame-like and object-oriented models for declarative knowledge. However,the proposed types of representation are not orientedat independent (without knowledge engineers) formationand modification of knowledge by domain experts.
For the formation of knowledge bases one can considerspecialized tools based on ontologies: Protégé, OntoEdit,GrOWL, Graphl, RDFGravity, WebVOWL, Ontolingua,OilEd, WebOnto, WebODE [9]. However, they usuallyimplement an object-oriented paradigm of knowledgerepresentation, incomprehensible to most domain ex-perts. A question of their integration with a problemsolver and UI also remains open. In accordance with theknowledge representation model, an appropriate mecha-nism for implementing the solver (reasoner) is proposed.If there are several models supported by the system,respectively, several solver implementation mechanismsand languages are supported. E.g., the SWORIER systemuses a reasoning mechanism based on ontologies andrules. Such solutions, on the one hand, are aimed atgiving the possibility of choosing the most adequateknowledge representation model and the correspondingsolver, but on the other hand, the transparency of suchsystems remains quite low.
The support of UI development is carried out in severalways. The developer is offered a set of tools providedby the toolkit, for example, [7]. This may be a special-ized programming language or tools similar to interfacebuilders, offered by various CASE-tools: a set of WIMPinterface elements that a user can define, specify their
properties and associate them with commands (user and/ or solver actions) and / or data (input or output). Theinteraction scenario in this case is embedded into thesolver. Interface development can be carried out usingthe language in which the solver is designed. Interactionwith different libraries provided by the toolkit is possible.
Thus, the most flexible tool for KBS implementationare specialized tool systems, as they allow one to imple-ment different classes of KBS. However, the problem ofthe viability of this class of systems is still far from afinal solution. Therefore, the search for new, improvedmechanisms for viability improvement of such systemsremains an urgent task.
III. BASIC PRINCIPLES OF KBS VITALITY
The viability of SS and KBS in particular is largelydetermined by their transparency. One of the main at-tributes of a transparent SS is clarity for interest groups.For KBS such groups are:
• domain experts who are responsible for the developmentand maintenance of KB,
• programmers who create and maintain a solver,• interface designers who implement the UI of a solver and
the UI for KB editors.
For KBS, it is fundamentally important to use relevantknowledge that must be formed and maintained bydomain experts or inductively (but in the latter case itsrepresentation should be intelligible to experts). This ispossible only if the knowledge representation languageis focused on the class of problems to be solved, and itsterminology is familiar to experts. To ensure the trans-parency of the solver, its structure and modules shouldbe clear to the maintainer. This is possible if most of thesolver is presented declaratively (which allows to controlsolvers with the help of editors), and domain knowledgeis not included in the solver. UI transparency can beensured, firstly, by providing users with different typesof UI which suite the model for presenting informationmost appropriately, secondly, by separating the data fromthe logic of its processing and its presentation method.The latter also provides separate modification of each ofthe components.
To implement these requirements, the following basicsolutions are proposed:
• common principles for creating KBS components;• a two-level approach to the formation of components:
first, a structural declarative model (component ontology)is formed, then the necessary component of the KBS iscreated by it;
• unified language and editor for creating models of allcomponents;
• automatic generation of editors for creation of compo-nents basing on their models;
• implementation of instrumental and applied intelligentsystems as cloud services.
All proposed solutions are based on a model descrip-tion language that allows one to describe arbitrary modelsoriented and adapted to the terminology of developers,
22
with the transition from general concepts to detailedones. The models of the components of intelligent ser-vices are formed in the model description languageand are represented in the form of a connected markedrooted hierarchical binary digraph. The markup definesthe semantics for the rules of formation (creation andmodification) of components, imposing restrictions ontheir structure and content [10], [11].
A. Development and maintenance of the knowledge base
In accordance with the two-level approach to theformation of components of the KBS, at the first stage aspecialized model of knowledge (data) representation isformed – the ontology of knowledge, which takes intoaccount the specifics of the organization of knowledgeand data in a given domain. Further, according to themodel of knowledge (data), the component editor gener-ator builds an editor of the knowledge base / database(see Fig. 1). Domain experts have the opportunity toform knowledge and data bases in terms of their conceptsystems but not in terms of some fixed knowledge anddata representation language.
Figure 1. Knowledge base formation process.
B. Development and maintenance of problem solver
The problem solver is a set of agents that interactwith each other by the exchange of messages. In ac-cordance with the two-level approach, developers areoffered unified agent and solver models for all services.To organize the launch of solvers with specific sets ofinput and output data, the cloud service model is alsodefined. To increase the transparency of the imperativeparts of the agent and of the message template after thedescription of their declarative part is specified, theirsource code sketch in the Java language is generated.The developed imperative code is associated with the
corresponding vertex of the agent (or message template)model.
C. UI development and maintenance
The development of an interface of intelligent ser-vices implies the development of a web interface. Theinterface design is based on the Model-View-Controller(MVC) pattern. Its fundamental principle consists in theseparation of data, the logic of its processing and theway it is presented in order to provide independentmodification of each component. The projection of thispattern on the interface model is as follows. The Modelcomponent includes: an abstract UI model containing adescription of the structure of standard WIMP interfaceelements (simple and container ones) and a way fortheir recursive organization into a single nested structure,as well as a software interface (API) for generatingfragments of abstract interfaces. The View componentis implemented by the system View agent. Its mainfunction is to create a description of a specific interfacebased on the description of an abstract interface and onrules of mapping from later to former. The Controllercomponent is represented by agents which play therole of an Interface Controller being a part of variousproblem solvers. These agents interact with the Viewagent by exchanging messages using specific templatesand implement necessary processing logic.
IV. CONCEPTUAL ARCHITECTURE OF DEVELOPMENTTOOLS
A comprehensive solution to the problem of theintelligent service viability also means providing theviability of the tools with which the service is createdand maintained. As a rule, the toolkit is maintained by itsdevelopers, but it must also be maintained by the KBSdevelopers [12]. For the successful implementation ofthis requirement, a three-tier toolkit architecture is pro-posed, consisting of the Toolkit Core, the Basic Toolkitand the Extensible Toolkit.
The Toolkit Core implements the basic principle of theconstruction of all components and includes the modeldescription language, the model editor, the generator ofcomponent editors. The declarative language for modeldescription is used to create component models, re-gardless of component’s purpose. The component modeleditor allows developers to create models in simple andconvenient way. The generator is designed to automati-cally build declarative component editors by componentmodels (ontologies). It is responsible for generating theUI and the component formation scenario which includeschecking the context conditions specified in the modeland the completeness of the component. The Toolkit Coreis sufficient for creating and controlling all intelligentservice declarative components by their models.
23
The main task of the Basic Toolkit is to provide thedeveloper with a set of tools for creating software com-ponents, assembling and binding them with informationcomponents, launching, and organizing infrastructure atall levels of the toolkit. Since all components are formedaccording to their structural declarative models, this levelof toolkit includes component editors that are generatedautomatically. In addition to the elements mentionedabove, it contains external software for creating theintelligent service UI and the imperative part of theaforementioned components.
The Extensible Toolkit is primarily intended for KBSdevelopers, who can expand it with new convenient toolsfor maintenance of KBSs developed by them and withspecialized or universal shells of expert systems. Theexpansion may be carried out using the Toolkit Core, theBasic Toolkit, as well as with the tools and instrumentalmechanisms of the Extensible Toolkit itself. This way,recursive use of its developed components is achieved.
The three-tier architecture forms the basis of theIACPaaS cloud platform (https://iacpaas.dvo.ru) [13],which is available for use by all developers of KBSsand their components. To date, portals of knowledge onmedicine, mathematics, autonomous uninhabited under-water vehicles, diagnostics of crops, information security,educational psychology, and programming technologyhave been created on the platform.
V. CONCLUSION
The paper considers mechanisms aimed at ensuring theviability of one class of software systems – systems withknowledge bases. Their main difference from systems of otherclasses is the presence of a knowledge base, which is subjectto continuous changes during the life cycle, and which must becreated and maintained by domain experts. The proposed so-lutions are based on the model description language developedby the authors, which provides the tools for model specificationin the form of connected labeled rooted hierarchical binarydigraphs with possible loops and cycles. The KBS componentswhich are built on the basis of the model have a unifiedrepresentation and internal storage format, and are providedwith a standardized and extensible set of software interfacesfor uniform access to them. Domain experts get the opportunityto form knowledge and data bases in terms of their conceptsystems but not in terms of fixed the knowledge and data rep-resentation language. The problem solver architecture includesdeclaratively represented software units, which can constitutea dynamic configuration, interact by message exchange, andthe structure of which is also described using a declarativemodel. A unified language and a uniform internal representationof both models and components, specified by them, allow theuse of common principles for editor generation basing on theKBS component type. All proposed ideas are implemented onthe IACPaaS cloud platform. Herewith, tool services providingsupport for the development technology are created on the sameprinciples as the applied services.
At the same time, the experience of platform usage and theavailability of user feedback has set a number of new scientificproblems, including the creation of language-oriented queries toknowledge bases, methods for creating adaptive user interfacesof various types for KBS problem solvers and knowledge
editors. Their solution will end up as additional increasing ofthe viability for this class of systems.
REFERENCES
[1] Pressman R.S. Software engineering: a practitioner’s approach. McGraw-Hill, 7th ed., 2010. 930 p. ISBN: 0073375977.
[2] Kryazhich O.A. Obespechenie zhiznesposobnosti informacii vo vremenipri ee obrabotke v SPPR [Ensuring the viability of the information duringits processing in decision support systems]. Matematicheskie mashiny isistemy [Mathematical Machines and Systems], 2015, no. 2, pp. 170–176.
[3] Chernikov B.V. Upravlenie kachestvom programmnogo obespecheniya[Software quality management], Moscow: Forum-Infra-M, 2012. 240 p.
[4] Yu-Cheng Tu. Transparency in Software Engineering. A thesis submittedin fulfillment of the requirements of Doctor of Philosophy in Electrical andElectronic Engineering. The University of Auckland. New Zealand, 2014.337 p.
[5] Rybina G.V. Intellektual’nye sistemy: ot A do YA. Seriya monografij vtrekh knigah. Kn. 3. Problemno-specializirovannye intellektual’nye sis-temy. Instrumental’nye sredstva postroeniya intellektual’nyh system [In-telligent systems: A to Z. A series of monographs in three books. Book3. Problem-specialized intelligent systems. Tools for building intelligentsystems], M.: Nauchtekhlitizdat, 2015. 180 p.
[6] Emmanuel C. Ogu, Adekunle, Y.A. Basic Concepts of Expert SystemShells and an Efficient Model for Knowledge Acquisition. Intern. J. ofScience and Research Intern. Journal of Science and Research (IJSR), IndiaOnline ISSN: 2319-7064, 2013, vol. 2, issue 4, pp. 554–559.
[7] Rybina G.V. Intellektual’naya tekhnologiya postroeniya obuchayushchihintegrirovannyh ehkspertnyh sistem: novye vozmozhnosti [Intelligent tech-nology for construction of tutoring integrated expert systems: new aspects].Otkrytoe obrazovanie [Open Education], 2017, vol. 21, no 4, pp. 43–57.
[8] Golenkov V., Gulyakina N., Grakova N., Davydenko I., Nikulenko V.,Eremeev A., Tarasov V. From training intelligent systems to training theirdevelopment tools. Otkrytye semanticheskie tekhnologii proektirovaniyaintellektual’nykh system [Open semantic technologies for intelligent sys-tems], 2018, pp.88–98.
[9] Ontology Tools. Available at: http://wiki.opensemanticframework.org/ in-dex.php/Ontology_Tools (accessed 2019, Jan).
[10] Gribova V.V., Kleshchev A.S., Moskalenko F.M., Timchenko V.A. A Two-level Model of Information Units with Complex Structure that Correspondto the Questioning Metaphor. Automatic Documentation and MathematicalLinguistics, 2015, vol. 49, no. 5, pp. 172–181.
[11] Gribova V.V., Kleshchev A.S., Moskalenko F.M., Timchenko V.A. AModel for Generation of Directed Graphs of Information by the DirectedGraph of Metainformation for a Two-Level Model of Information Unitswith a Complex Structure. Automatic Documentation and MathematicalLinguistics, 2015, vol. 49, no. 6, pp. 221–231.
[12] Musen M. The protégé project: a look back and a look forward. NewsletterAI Matters, 2015, vol. 1, iss. 4, pp. 4–12.
[13] Gribova V., Kleschev A., Moskalenko P., Timchenko V., Fedorischev L.,Shalfeeva E. The IACPaaS cloud platform: Features and perspectives.Computer Technology and Applications (RPC), 2017 Second Russia andPacific Conference on. IEEE, 2017, pp. 80–84.
МЕТОДЫИ СРЕДСТВА ПЛАТФОРМЫ IACPAAS ДЛЯСЕМАНТИЧЕСКОГО ПРЕДСТАВЛЕНИЯ ЗНАНИЙ ИРАЗРАБОТКИ ДЕКЛАРАТИВНЫХ КОМПОНЕНТОВ
ИНТЕЛЛЕКТУАЛЬНЫХ СИСТЕМ
В.В. Грибова, А.С. Клещев, Ф.М. Москаленко,В.А. Тимченко, Л.А. Федорищев, Е.А. Шалфеева
В работе обсуждается проблема обеспечения жизнеспо-собности интеллектуальных систем – систем с декларатив-ными базами знаний. Рассмотрены инструментальные про-граммные средства для разработки систем данного класса,реализующие механизмы повышения их жизнеспособности.Эти механизмы основаны на построении каждого компо-нента по его декларативной модели, специфицируемой наедином языке описания моделей.
Received 11.01.19
24
Methods and tools for ensuring compatibility ofcomputer systems
Vladimir Golenkov, Natalia Guliakina, Irina DavydenkoBelarussian State University
Informatics and RadioelectronicsMinsk, Belarus
[email protected], [email protected], [email protected]
Aleksandr EremeevNational Research University "MEI"
Moscow, [email protected]
Abstract—The paper discusses the main current prob-lems in the modern computer systems development, in par-ticular – the problem of ensuring information compatibilityof computer systems. An approach to their solution, basedon the use of the Open Semantic Technology for IntelligentSystem Design (OSTIS), is proposed.
Keywords—semantic computer system, semantic tech-nology, hybrid systems, computer systems compatibility,OSTIS Technology, SC-code, ontology
I. INTRODUCTION
Until now, traditional information technologies andartificial intelligence technologies have evolved indepen-dently of each other.
Now is the time for fundamental rethinking of theexperience of using and evolving traditional informationtechnologies and their integration with artificial intelli-gence technologies. This is necessary to eliminate a num-ber of shortcomings of modern information technologies.
The experience of using computer systems to automatevarious types of human activity shows that automationof disorder leads to even more confusion, and illiter-ate automation is worse than its absence. Moreover, ifautomation requires the use of methods and artificialintelligence, the consequences of illiterate automationcan be even more devastating.
This means that before proceeding with the automationof any activity (and, especially, with the use of artificialintelligence), it is necessary to build a qualitative formalmodel of this activity (that is, a sufficiently detailedholistic description of it, but without excesses).
In our report at the conference OSTIS-2018 [1] thekey property of intelligent systems was considered - theirlearnability, as well as those properties of intelligentsystems that provide a high level of learnability (flexi-bility, stratification, reflexivity).
In this paper, the currently key problem of the de-velopment of information technologies in general andof artificial intelligence technologies in particular, theproblem of ensuring information compatibility ofcomputer systems, including intelligent systems, will beconsidered.
The urgency of solving this problem is due to the factthat:
• informational compatibility of computer systemswill significantly increase the level of their learn-ability due to more effective perception of experi-ence (knowledge and skills) from other computersystems;
• it will be possible to significantly expand thediversity of the knowledge and skills used in thecomputer system without the need to develop spe-cial tools for their coordination. It also increasesthe level of learnability of computer systems andallows you to move to hybrid, synergistic computersystems;
• it will be possible to create collectives of computersystems, using universal principles of the organiza-tion of interaction between computer systems at themeaningful level;
• It will be possible not only to develop compat-ible computer systems, but also to automate theprocess of permanent support of computer sys-tems compatibility. The need for this support isdue to the fact that the compatibility of computersystems during their operation and evolution may beviolated. Consequently, there must be tools whichwill permanently restore the compatibility of com-puter systems in the conditions of their permanentchange;
• it will be possible to automate the process of per-manent support (restoration) of information com-patibility of computer systems not only with othercomputer systems, but also with their users;
• it will be possible to significantly reduce the de-velopment time of new computer systems usingthe permanently expanding library of reusablecomputer system components, which have differ-ent levels of complexity (up to typical embeddedsubsystems) and different types (typical embeddedknowledge, for example, ontologies, widely usedskills, in particular, programs, interface subsystems,providing messaging with external subjects in a
25
given external language).Lets consider the problems of information technology
development:• in the field of traditional computer systems;• in the field of intelligent systems;• in the field of informatization of scientific and
technical activities.
II. STATE AND PROBLEMS OF TRADITIONALINFORMATION TECHNOLOGIES
The current state of traditional information technolo-gies in general can be described as:• illusion of well-being;• illusion of omnipotence of financial resources in
solving complex technical problems;• "Babel" of various technical solutions, the compat-
ibility of which no one seriously thinks about;• lack of an integrated systems approach to automat-
ing complex types of project activities;• lack of awareness that the shortcomings of mod-
ern information technologies are of a fundamental,systemic nature.
The shortcomings of modern information technologiesinclude:
1) Diversity of syntactic forms of presentation ofthe same information, i.e. variety of semanti-cally equivalent forms (languages) of representa-tion (coding) of the processed information (knowl-edge) in the memory of computer systems. Thelack of unification of the representation of varioustypes of knowledge in the memory of moderncomputer systems leads:•• to the variety of semantically equivalent models
for problems solving (both procedural and non-procedural - functional, logical, etc.), i.e. toduplication of information processing modelsthat differ not in the essence of the methods ofproblems solving, but in the form of presentationof the processed information and the form ofrepresentation of methods (skills) of solving ofvarious problems classes;
•• to the duplication of semantically equivalentinformation components of computer systems;
•• to the variety of forms for the technical im-plementation of each model used for problemssolving;
•• to the semantic incompatibility of computer sys-tems and, consequently, to the high complexityof their integration into systems of a higher levelof hierarchy, which requires additional effortsto translate (convert) information shared by dif-ferent integrable systems and, therefore, signif-icantly limits the effectiveness of joint problemsolving by a team of interacting computer sys-tems. The complexity of the integration process
can be significantly reduced by transition of theintegrated computer systems to some uniformform, since in this case the integration can becarried out in a universal and automated way;
•• to a significant decrease in the effectivenessof the use of the method of computer sys-tems component design based on libraries ofreusable components (especially when it comesto "large" components, in particular, typical sub-systems) [2].
2) Insufficiently high degree of learnability of moderncomputer systems during their operation, resultingin a high complexity of their maintenance andimprovement, as well as their insufficiently longlife cycle.
3) The lack of opportunity for experts to really in-fluence on the quality of the developed computersystems. The experience of complex computersystems development shows that the mediation ofprogrammers between experts and projected com-puter systems substantially distorts the contributionof experts. When developing next-generation com-puter systems, it is not programmers who shoulddominate, but experts who are able to accuratelystate their knowledge.
4) The lack of semantic (sense) unification of theinterface activity of users of computer systems,which, together with the variety of forms for im-plementing user interfaces, leads to serious over-head costs for learning of user interfaces of newcomputer systems.
5) Computer system documentation is not an im-portant component of the computer system itself,determining the quality of operation of this system,resulting in an insufficiently high efficiency ofcomputer system operation due to incomplete andinefficient use of the capabilities of the computersystem being operated.
To overcome these shortcomings is possible onlythrough a fundamental rethinking of the architectureand principles of the organization of complex computersystems. The basis of this rethinking is the eliminationof the diversity of forms of representation (coding) ofinformation in the memory of computer systems.
The result of this rethinking should be a new stage inthe development of information technology.
Overcoming the shortcomings of modern computersystems involves:
• unification of the information processed;• functional unification (unification of information
processing principles).26
III. PROBLEMS OF ARTIFICIAL INTELLIGENCETECHNOLOGIES DEVELOPMENT
Expansion of computer systems applications leads tothe expansion of the variety of automated activities –management of various types of enterprises, managementof organizations, management of complex technical sys-tems, multisensory integration and primary analysis ofnon-verbal information, recognition, design of artificialobjects of various types, design of business processsystems aimed at reproduction of the designed artificialobjects, communication with users (on natural languagesin text and speech form, using the means of cognitivegraphics), user learning, comprehensive information ser-vices for users.
In turn, the expansion of the variety of automatedactivities leads to the expansion of the variety of typesof problems solved, types of methods and tools forproblems solving, types of information used (types ofknowledge).
For example, increasing the level of automation ofvarious enterprises leads to a knowledge-oriented or-ganization of their activities, and in the future – to aknowledge-oriented economy. This means that knowl-edge management tools become the basis of enterpriseautomation.
From this, in turn, it follows that in perspective enter-prise management systems it is necessary to move fromdatabases that provide a presentation of fairly simple(factographic) types of knowledge to knowledge bases,which may include knowledge of the most diverse types.
A. The evolution of computer systems
Thus, the expansion of the field of application ofcomputer systems requires a transition from traditionalcomputer systems to systems focused on processing awide variety of structured information, as well as on solv-ing more and more complex problems. Consequently, thetransition from traditional computer systems to intelligentsystems is inevitable. Moreover, this transition has longbeen happening. This is confirmed by such directions ofevolution of computer systems as:• the transition from the dominance of programs to
the dominance of the processed information, i.e.,data-driven computer systems;
• from semi-structured data to structured data anddata independent of the programs that process thisdata, i.e., to databases;
• from data to knowledge by expanding semantictypes of processed information, and further to com-puter systems, managed by structured knowledge,and to computer systems, managed by knowledgebases;
• transition from non-context problem solving, theinitial data for which are a priori exactly specified,to problem solving with the active use of the context
of these problems, i.e. knowledge of the subjectdomain in which the task is being solved;
• transition from procedural low-level programminglanguages to high-level procedural programminglanguages, and to non-procedural programming lan-guages (functional, logical);
• transition from sequential to parallel programs;• transition from synchronous information processing
to asynchronous;• transition from programs to calculations, to "soft"
calculations (fuzzy logic, genetic algorithms, artifi-cial neural networks);
• transition from data-oriented programs, where thedata structuring is determined by the correspondingprograms, to programs oriented to database process-ing and further knowledge bases processing;
• transition from address memory to associative mem-ory;
• transition from linear memory to non-linear (recon-structable, reconfigurable, graph-dynamic) memory,in which information processing is reduced not onlyto a change in the state of the elements in thememory, but also to a change in the configurationof the connections between them;
• transition from traditional computer systems tocomputer systems capable of solving a wide varietyof complex (difficult to formalize) problems and,including intelligent problems, to computer systemswith a hybrid well-structured high-quality knowl-edge base, with a hybrid problem solver, with ahybrid (multimodal) interface (both verbal and non-verbal);
• transition from non-learnable computer systems tolearnable.
Consequently, the intellectualization of computer sys-tems is the natural direction of their evolution.
The modern most actively developed areas of devel-opment of intelligent systems include:
• knowledge management and ontological engineer-ing [3], Semantic Web [4];
• formal logic (strict, fuzzy, deductive, inductive, ab-ductive, descriptive, temporal, spatial, etc.);
• artificial neural networks, Bayesian networks, ge-netic algorithms (Machine learning in the narrowsense);
• computer linguistics (NLP), semantic analysis ofnatural language texts;
• speech processing, semantic analysis of voice mes-sages;
• image processing - technical vision, semantic imageanalysis;
• multi-agent systems, collectives of intelligent sys-tems [5], [6], [7];
• hybrid intelligent systems, synergistic intelligentsystems [8].
27
B. The current state of artificial intelligence technology
Despite the presence of serious scientific results inthe field of artificial intelligence, the rate of evolutionof the intelligent systems market is not so impressive.
There are several reasons for this:• there is a big gap between scientific research in
the field of artificial intelligence and the creationof high-quality technologies for the development ofintelligent systems. Scientific research in the fieldof artificial intelligence is mainly focused on thedevelopment of new methods for solving intelligentproblems;
• these researches are scattered and not aware ofthe need for their integration and the creation ofa general formal theory of intelligent systems, i.e.there is a "babel" of various models, methods andtools used in artificial intelligence in the absenceof awareness of the problem of ensuring theircompatibility. Without solving this problem, neitherthe general theory of intelligent systems nor, there-fore, the complex technology of intelligent systemsdevelopment available to engineers and experts canbe created;
• the specified integration of models and methods ofartificial intelligence is very complex, since it isinterdisciplinary in nature;
• intelligent systems as objects of design have asignificantly higher level of complexity comparedto all the technical systems with which humanityhave had a deal;
• as a consequence of the above, there is a big gapbetween scientific research and engineering practicein this area. This gap can be filled only by creatingan evolving technology of intelligent systems devel-opment, the creation of which is carried out throughactive cooperation of scientists and engineers;
• the quality of development of applied intelligentsystems depends to a large extent on the mutualunderstanding of experts and knowledge engineers.Knowledge engineers, not knowing the intricaciesof the applied area, can introduce serious errorsinto the developed knowledge bases. The media-tion of knowledge engineers between experts andthe knowledge base being developed significantlyreduces the quality of the developed intelligentsystems. To solve this problem, it is necessarythat the knowledge representation language in theknowledge base be convenient not only to theintelligent system and knowledge engineers, butalso to experts.
The current state of artificial intelligence technologycan be described as follows:• There is a large set of proprietary artificial intelli-
gence technologies with appropriate tools, but thereis no general theory of intelligent systems and, as
a result, there is no overall integrated technologyfor intelligent systems design (see Artificial GeneralIntelligence conference [9]);
• Compatibility of particular technologies of artificialintelligence is practically not implemented, andmoreover, there is no awareness of such a need.
The development of artificial intelligence technolo-gies is significantly hampered by the following socio-methodological circumstances:• High social interest in the results of work in the
field of artificial intelligence and the great com-plexity of this science gives rise to superficialityand untidiness in the development and advertisingof various applications. Serious science is mixedwith irresponsible marketing, conceptual and termi-nological negligence and illiteracy, throwing in newabsolutely unnecessary effective terms that confusethe essence of the matter, but create the illusion offundamental novelty.
• The interdisciplinary nature of research in the fieldof artificial intelligence significantly complicatesthese researches, because work at the junctions ofscientific disciplines requires high culture and skills.
C. Directions of development of artificial intelligencetechnologies
To solve the above problems of the development ofartificial intelligence technology:• Continuing to develop new formal models for in-
telligent problems solving and to improve existingmodels (logical, neural network, production), it isnecessary to ensure compatibility of these modelsboth among themselves and with traditional modelsfor problems solving that were not included in thenumber of intelligent problems. In other words, weare talking about the development of principles forthe organization of hybrid intelligent systems thatprovide solutions to complex problems that requirejoint in unpredictable combinations of the mostdiverse types of knowledge and the most diversemodels for problems solving.
• A transition is needed from the eclectic constructionof complex intelligent systems using various typesof knowledge and various types of problem solvingmodels to their deep integration, when the samerepresentation models and knowledge processingmodels are implemented in different systems andsubsystems in the same way.
• It is necessary to reduce the distance between themodern level of the theory of intelligent systemsand the practice of their development.
• It is necessary to significantly increase the level ofconsistency of actions of persons involved in theprocess of continuous improvement of knowledgebases.
28
• It is necessary that the systems themselves, and notjust their developers, actively participate in solvingthis compatibility problem of intelligent systems.Systems themselves must take care of maintainingtheir compatibility with other systems in the contextof the active change of these systems through themechanism of automated coordination of the con-cepts used between intelligent systems.
IV. PROBLEMS OF DEVELOPMENT OF METHODS ANDTOOLS OF SCIENTIFIC ACTIVITY INFORMATIZATION
It is obvious that the highest form of informationactivity is scientific activity and, therefore, the high-est level of development of computer systems are thesystems that are directly and actively involved in thisactivity. Scientific activity is aimed at improving thequality of our knowledge about the world around usand, therefore, is associated with the analysis, processingand systematization of this knowledge. It is obviousthat if computer systems aimed at automating scientificactivities understand the scientific knowledge they pro-cess and, therefore, will become not passive performers,but scientific partners who are able to independentlyanalyze, systematize scientific knowledge and use themin various problems solving then the level of automationof scientific activity will be significantly increased.
The most important restraining factors of scientific andtechnological progress at present are:• diversity ("babel") of both natural and formal lan-
guages used to present the results of scientific andtechnical research;
• binding scientific and technical texts to naturallanguages (monographs, reports, articles);
• fundamental contradiction between the principlesof the evolution of natural languages as the mainmeans of communication and the requirements forscientific and technical languages.
To solve these problems we need:• to build a strict formal system of scientific and
technical languages;• to build a clear connection between scientific and
technical and natural languages;• to ensure the design of scientific and technical texts
in compatible formal languages that are understand-able and convenient for both people and computersystems;
• to provide support for the evolution of this multi-language complex.
The most important direction of increasing the effec-tiveness of scientific and technical activities (and, in par-ticular, increasing the rate of scientific and technologicaldevelopment) is the transition from the traditional versionof the results of this activity (in the form of reports, ar-ticles, monographs, reference books) to the presentationof scientific and technical information in the form of an
encyclopedic systems of interconnected knowledge baseson various scientific and technical disciplines. The formalresult of any scientific discipline should be a knowledgebase reflecting the current state of this discipline. For ap-plied scientific disciplines, an additional result should bea computer-aided design system for designing artificialsystems of the corresponding class that is accessible toengineers.
The idea of the difficulties of such a transition isgreatly exaggerated, since modern tools of knowledgeengineering are ready for the implementation of suchprojects. This is prevented by:• fear of the new, unusual;• need to revise the organization of scientific and
technical activities.But the perspective is a transition to a qualitatively new
level of culture of scientific and technological progress.The social significance of this transition is as follows:• The rate of evolution of scientific knowledge will
significantly increase due to the fact that the ob-tained scientific knowledge is presented in a formconvenient for both people and computer systems,as well as by automating their integration, analysis,structuring and coordination of various points ofview.
• The efficiency of the use of scientific knowledge inthe developed computer systems will significantlyincrease, due to the fact that there is no need for thestep of formalizing this knowledge to be includedin the knowledge bases.
• The possibility of direct participation of students inimproving the knowledge that corresponds to theacademic disciplines they study will significantlyimprove the quality of such learning, since promotesindividual, active and systematic learning of theeducational material.
The main problem of the development of scientific andtechnical activities and, accordingly, of its informatiza-tion is the need for deep convergence of various scien-tific disciplines, as discussed in a number of works [10],[11].
An important problem is also the reduction of timeand laboriousness in organizing informational interactionbetween scientists in the agreement of points of view,in the joint implementation of any research, in the jointwork on articles or monographs, in reviewing.
It should be remembered that any point of view al-ways has shortcomings (incompleteness, fuzziness, etc.).Therefore, it is methodologically necessary to move fromthe practice of confronting points of view to the practiceof integrating points of view (including those that seemto be alternative, contradictory). Only in the developmentof complex systems can a synergistic effect be achieved,which is based on compensation for the shortcomings ofsome points of view by the advantages of others.
29
This is how the organization of a collective creativeprocess should be arranged. Automating such a processinvolves fixation of a multiplicity of points of view andmanaging the process of reconciling these points of view.
V. THE PROPOSED APPROACH TO SOLVINGPROBLEMS THAT HINDER THE FURTHER EVOLUTION
OF COMPUTER SYSTEMS AND TECHNOLOGIES -STANDARDIZATION OF INFORMATION
REPRESENTATION AND PROCESSING MODELS
Analysis of the problems of the evolution of computersystems of different levels of complexity, different levelsof learnability and intelligence, of different purposesshows that the curse of the “babel” and, as a result,incompatibility, duplication and subjectivity of coordi-nated information resources and models of processingthem haunts us everywhere:• and in the development of traditional computer
systems;• and in the development of artificial intelligence
technologies;• and in the development of methods and tools of
informatization of scientific and engineering activ-ities.
Considering the problem of ensuring the compatibilityof information resources and models of their processing,we should talk about various aspects of solving thisproblem:• about ensuring compatibility between various com-
ponents of computer systems, as well as betweencomplete computer systems that are part of com-puter systems teams;
• about compatibility, i.e. high level of mutual under-standing between different computer systems andtheir users;
• about interdisciplinary compatibility, i.e. conver-gence of different areas of knowledge;
• about the methods and means of continuous moni-toring and restoring compatibility in the conditionsof intensive evolution of computer systems and theirusers, which often violates the achieved compatibil-ity (consistency) and requires additional efforts torestore it.
A. Directions of the evolution of computer systems
In the evolution of computer systems can be distin-guished two general directions.
First Direction is• expansion of the set and variety of problems
solved by a computer system;• increase the complexity of these problems down
to difficultly formalized (difficultly solvable) prob-lems, intelligent problems solved in the conditionsof incompleteness, inaccuracy, vagueness, etc .;
• increase quality of problem solving either by moreefficient use of known models for problems solving(for example, by developing better algorithms), orby using fundamentally new models for problemssolving;
• extension the variety of information (knowledge)used;
• extension the variety of used problems solvingmodels.
Obviously, the expansion of the set of solved problemsin the conditions of a large but always finite memory ofa computer system makes the transition from particularmethods and models for solving problems to their gen-eralizations (or, as D.A. Pospelov noted, from bundle of"keys" to a set of "lockpicks").
It is also obvious that the variety of types of problemssolved by computer systems, the variety of models usedfor problems solving leads:• to integrated information resources;• to integrated problem solvers;• to integrated computer systems;• to computer system teams.The problem here is not the integration itself, but its
quality. Integration may be eclectic if the compatibilityof the integrable components is not ensured, and in thecase of such compatibility integration may lead to a newquality, to an additional expansion of the set of solvedproblems. This will mean a transition from eclecticismto hybridity, synergy.
The second general direction of the evolution ofcomputer systems is the increase in their learnabilityand, as a result, the rate of their evolution.
Learnability of computer system is determined by:• labor intensity and the pace of acquisition (expan-
sion) and improvement of actively used knowledgeand skills;
• level of restrictions imposed on the type of ac-quired and used knowledge and skills (in fact, theseare restrictions on the set of all those problemsthat can in principle be solved by a given computersystem).
In turn, the labor intensity and rate of expansion andimprovement of the knowledge and skills of a computersystem is determined by:• flexibility – the variety and laboriousness of possi-
ble changes made to the system in the process ofreplenishing the system with new knowledge andskills and improving already acquired knowledgeand skills;
• stratification – a clear separation of the system intohierarchy levels that are rather independent of eachother, i.e. the possibility of localizing fragments ofa computer system, without going beyond of whichit is apriori possible to analyze the effects of certainchanges in the system;
30
• reflexivity –– the ability to analyze one’s own stateand one’s activity;
• hybridity - the ability to acquire and use a wide(and ideally unlimited) variety of knowledge andskills;
• level of self-learnability - the level of activity,independence, purposefulness in the process oftheir learning, i.e. the level of ability to learnwithout a teacher, the level of automation of theacquisition of new knowledge and skills, as wellas the improvement of already acquired knowledgeand skills;
• compatibility – integration complexity;• the ability to continuously monitor and maintain
its compatibility with other computer systems andwith its users in the context of the intensive evolu-tion of these computer systems and their users.
Compatibility (integration complexity) of computersystems can be considered in two aspects:• in the aspect of deep integration of computer
systems, which involves the transformation of sev-eral computer systems into one consistent computersystem by combining information and functionalresources of integrable computer systems;
• in the aspect of converting several computer sys-tems into team of interacting computer systems,capable of jointly corporate solving of complexproblems.
Compatibility (complexity of integration) of computersystems is determined by:• compatibility of various types of information
(knowledge) stored in the memory of a computersystem;
• compatibility of various problem solving models;• compatibility of embedded (including typical) sub-
systems that are part of computer systems;• compatibility of external information entering the
computer system with information stored in thememory of a computer system (the laboriousnessof understanding external information - translation,immersion, concepts aligning);
• communication (including semantic) compatibilitywith users and with other computer systems.
The most important form of computer system learningis the acquisition of new knowledge and skills in the"ready" form, i.e. in the form of some sign structuresentered into the memory of a computer system, since theacquisition of knowledge and skills from external reliablesources requires significantly less time compared to theiracquisition on its own, based on its own experience andits own mistakes. But in order for this form of learningto be effective, it is necessary to simplify and formalizeas much as possible the mechanism (procedure) of im-mersing new knowledge in the memory of a computersystem.
To solve this problem, the creation of a convenientmethod for coding various types of information in thememory of a computer system is of key importance.
Since the main channel for learning computer systemsis the acquisition of knowledge and skills from othersubjects – from other computer systems and from users(from developers-teachers and from end users). Conse-quently, the level of learnability of computer systems isalso determined by the level of its compatibility withthese external subjects themselves, with the knowledgeand skills acquired by it, i.e. the degree of how thecomputer system, together with the subjects with whichit exchanges information, solves the problem of the"babel".
B. The essence of the proposed approach
The essence of our approach to solving the prob-lems of the evolution of computer systems is, firstly,to combine all the above directions of the evolution ofcomputer systems (both general directions and particularones) and, secondly, to interpret the problem of providingcompatibility types of knowledge, various models forsolving problems, various computer systems as the keyproblem of the evolution of computer systems, whosesolution will greatly simplify the solution of many otherproblems.
For example, without ensuring the compatibility of in-formation resources used in different computer systems,as well as information resources representing knowledgeof various semantic types, it is impossible:
• neither to create computer system teams capableof coordinating their actions while cooperativelysolving complex tasks;
• neither to create hybrid computer systems that arecapable of using various combinations of differenttypes of knowledge and different models of prob-lems solving when solving complex problems;
• neither to use the component design methodologyof computer systems at all levels of the hierarchyof designed systems.
What kind of informational compatibility and mutualunderstanding (including between specialists) can we talkabout in the presence of terrifying conceptual and ter-minological messiness, terminological pseudo-creativity,including, in the field of computer science.
Speaking about compatibility of computer systemsand their components, as well as compatibility of com-puter systems with users, we should note the ambiguityof the interpretation of the term “compatibility”. In thisregard, it should be distinguished:
• compatibility as one of the learning factors, likeability to quickly increase the level of consistency(integration, mutual understanding). Compare learn-ing as ability to rapidly expand knowledge and
31
skills, but not characterizing the volume and qualityof acquired knowledge and skills;
• compatibility as a characteristic of the achievedlevel of consistency (integration, mutual under-standing).
Similarly, the intelligence of a computer system, onthe one hand, can be interpreted as level (volume andquality) of acquired knowledge and skills, and on theother hand, as ability to rapidly expand and improveknowledge and skills, i.e. as speed enhance knowledgeand skills.
In addition, one should speak not only about theability to rapidly increase the level of consistency andnot only about the level of consistency achieved, but alsoabout the process of increasing the level of consistencyand, above all, about the permanent restoration process(support maintaining the level of consistency achieved,since during the evolution of computer systems and theirusers (i.e., in the course of expanding and improving thequality of their knowledge and skills), their consistencymay decrease.
C. Semantic unification of computer systems
The main factor in ensuring the compatibility ofvarious types of knowledge, various models of problemsolving and various computer systems in general is• unification (standardization) of information repre-
sentation in the memory of computer systems;• unification of the principles of organization of in-
formation processing in the memory of computersystems.
The unification of the information representation usedin computer systems implies:• syntactic unification of the information used - the
unification of the form of representation (coding) ofthis information. It should be distinguished:•• coding information in the memory of a com-
puter system (internal presentation of informa-tion);
•• external presentation of information ensuringthe unambiguous interpretation (understanding,interpretation) of this information by differentusers and different computer systems;
• semantic unification of the information used, whichis based on the agreement and exact specificationof all (!) used concepts using a hierarchical systemof formal ontologies.
It is important to note that competent unification(standardization) should not limit the creative freedomof the developer, but guarantee the compatibility of itsresults with the results of other developers. We alsoemphasize that the current version of any standard is nota dogma, but only a basis for its further improvement.
The goal of a quality standard is not only to ensure thecompatibility of technical solutions, but also to minimize
duplication (repeating) of such solutions. One of the mostimportant quality criteria of a standard is nothing excess.
standart= knowledge of the structure and principles of
functioning of artificial systems of the correspondingclass
= ontology of artificial systems of a certain class= theory of artificial systems of a certain class
Standards, like other knowledge important to human-ity, must be formalized and must be constantly improvedusing special intelligent computer systems that supportthe process of standards evolution by reconciling differ-ent points of view.
VI. THE STANDARD OF SEMANTIC REPRESENTATIONOF INFORMATION IN THE MEMORY OF A COMPUTER
SYSTEM
A. Unification of the internal presentation of informationin computer systems
The objective guideline for unification of informationrepresentation in the memory of computer systems andthe key to solving many problems of the evolution ofcomputer systems and technologies is formalization ofthe sense of the information being presented.
According to V. V. Martynov [12], «virtually everyhuman thought activity (not only scientific), as manyscientists believe, uses an internal semantic code, whichis translated from a natural language and from which itis translated into a natural language. The amazing abilityof a person to identify a huge variety of structurallydifferent phrases with the same meaning and the abilityof remember the meaning outside of these phrasesconvinces us of this.»
We also give the words of I.A. Melchuk [13]:« The idea was the next – the language should be
described as follows: one should be able to write downthe meanings of the phrases. Not phrases, but theirmeanings, which is separate. Plus build a system thatbuilds the meaning of the phrase. This is the area orthe turn of research in which the intuition of a capablelinguist works best: how to express this meaning in agiven language. This is what linguists are taught for ..
The linguistic meaning of a scientific text is not at allwhat you, reading it, extract from it. This, very roughlyspeaking, is an invariant of synonymous paraphrases. Youcan express the same meaning by so many. When yousay, you can say in different ways: “Now I pour youwine”, or: “Let, I will offer you wine”, or: “Shouldwe drink a glass of wine?”, - all this has the samemeaning. And here you can think of how to record thismeaning. Exactly it. Not a phrase, but a meaning. And itis necessary to work from this sense to real phrases. Thesyntax there is also needed by the way, but it is needed
32
only by the way, it can be neither the final goal, nor thestarting point. This is an intermediate case. » [14].
The clarification of the principles of semantic rep-resentation of information is based, firstly, on a clearcontrast between the internal language of a computersystem used to store information in computer memory,and external languages of a computer system usedfor communication (exchange messages) of a computersystem with users and other computer systems (senserepresentation is used exclusively for the internal rep-resentation of information in the memory of a computersystem), and, secondly, to possibility of simplificationof the computer system internal language syntax whileproviding versatility by excluding from such an internaluniversal language means providing a communicationfunction language (m. e. messaging).
For example, for the internal language of a com-puter system, such communication tools of the languageas conjunctions, prepositions, dividers, limiters, declen-sions, conjugations, and others are superfluous.
External languages of a computer system can be bothclose to its internal language, and very far from it (as,for example, natural languages).
Sense is a abstract sign construct belonging to theinternal language of a computer system, being the invari-ant of the maximum class of semantically equivalent signconstructions (texts) belonging to different languages andsatisfying the following requirements:
• universality – the ability to present any informa-tion;
• absense of synonymy signs (multiple occurrenceof characters with the same denotates);
• absense of duplication of information in the formof semantically equivalent texts (not to be confusedwith logical equivalence);
• absence of homonymous signs (including pro-nouns);
• absence of internal structure of signs (atomiccharacter of signs);
• absense of declensions, conjugations (as a resultof the absence of the internal structure of signs);
• absense of fragments of a sign construct, whichare not signs (separators, delimiters, etc.);
• distinguishing of connection signs, the compo-nents of which can be any signs with which connec-tion signs are associated with syntactically definedincidence relations.
The consequence of these principles of the semanticrepresentation of information in the memory of a com-puter system is that the entity signs included in the se-mantic presentation of information are not names (terms)and, therefore, are not tied to any natural language and donot depend on subjective term addictions of various au-thors. This means that from the collective development of
the semantic representation of any information resourcesterminological disputes are excluded.
The consequence of these principles of sense rep-resentation of information is also the fact that theseprinciples lead to non-linear sign structures (graph struc-tures), which complicates the implementation of com-puter system memory, but significantly simplifies itslogical organization (in particular, associative access).
The nonlinearity of the sense representation of infor-mation is due to the fact that:
• each described entity, i.e. an entity that has acorresponding sign can have an unlimited numberof connections with other described entities;
• each described entity in the sense representationhas a single sign, because synonymy of signs isprohibited here;
• all connections between the described entities aredescribed (reflected, modeled) by the connectionsbetween the signs of these described entities.
The essence of the universal sense representationof information can be formulated in the form of thefollowing provisions:
• Sense sign construction is interpreted as a set ofsigns, which are one-to-one designating differententities (denotations of these signs) and a set ofconnections between these signs;
• Each connection between signs is interpreted, onthe one hand, as a set of signs connected by thisconnection, and, on the other hand, as a description(reflection, model) of the corresponding connection,which connects the denotations of the specifiedsigns or the denotation of some signs directly toother characters, or these signs themselves. Anexample of the first type of connection betweensigns is the connection between signs of materialentities, one of which is part of the other. Anexample of the second type of connection betweensigns is the connection between the sign of the setof signs and one of the signs belonging to this set,as well as the connection between the sign and thefile sign, which is an electronic reflection of thestructure of the representation of the specified signin external sign structures. Examples of the thirdkind of connection between signs are the connectionbetween synonymous signs;
• The denotats of characters can be (1) not onlyspecific (constant, fixed), but also arbitrary (vari-ables, non-fixed) entities that "run through" varioussets of signs (possible values), (2) not only real(material), but also abstract entities (for example,numbers, points of various abstract spaces), (3) notonly "external" , but also "internal" entities, whichare sets of signs that are part of the same signstructure.
33
The key property of the sense representation of infor-mation language is the uniqueness of the information rep-resentation in the memory of each computer system, i.e.,the absence of semantically equivalent sign constructionsbelonging to the sense language and stored in one sensememory. At the same time, the logical equivalence ofsuch sign constructions is allowed and used, for example,for a compact representation of some knowledge storedin the sense memory.
However, the logical equivalence of the constructionsstored in the memory should not be carried away, becauselogically equivalent sign constructions are represen-tations of the same knowledge, but with the help ofdifferent sets of concepts. In contrast, semanticallyequivalent sign constructions are the representation ofthe same knowledge with the help of the same concepts.It is obvious that the variety of possible options for therepresentation of the same knowledge in the memory of acomputer system significantly complicates the problemssolving. Therefore, by completely eliminating semanticequivalence in semantic memory, it is necessary to striveto minimize logical equivalence. For this, a competentconstruction of a system of used concepts in the formof a hierarchical system of formal ontologies [15] isnecessary.
An important step in creating a universal formalmethod of sense coding of knowledge was developed byV.V. Martynov Universal Semantic Code (USC) [12].
As the standard of the universal sense representationof information in the memory of computer systemswe have proposed SC-code (Semantic Computer Code).Unlike USC of V.V. Martynov if, firstly, is non-linear innature and, secondly, is specifically focused on codinginformation in the memory of computers of a newgeneration, focused on the development of semanticallycompatible intelligent systems and called semantic as-sociative computers. Thus, the main leitmotif of theproposed sense presentation of information is the orienta-tion to the formal memory model of a non-Von-Neumanncomputer designed for the implementation of intelligentsystems using the sense representation of information.The features of this representation are as follows:
• associativity;• all information is enclosed in a connections config-
uration, i.e. processing information is reduced to thereconfiguration of connections (to graph-dynamicprocesses);
• transparent semantic interpretability and, as a result,semantic compatibility.
Implicit binding to Von Neumann computers is presentin all known knowledge representation models. Oneexample of such a dependency is, for example, theobligatory naming of the objects being described.
B. Syntax of SC-code
The universality of the SC-code allows using it todescribe any objects. This object can be any language ofcommunication with users (including natural language),as well as the SC-code itself. The syntax of the SC-codeis represented as the corresponding formal ontology. Thekey concepts of the subject domain that are described(specified) by the mentioned ontology are:
sc-element= atomic fragment of the sign construction stored in
the memory and belonging to the SC-codesc-nodesc-connectorsc-edge= non-oriented sc-connectorsc-arc= oriented sc-connectorbase sc-arcincidence of sc-connector*incidence of incoming sc-arc*
Within the specified domain, the class of all possiblesc-elements is the maximum class of studied objectsstudy, the concepts sc-node, sc-connector, sc-edge, sc-arc, base sc-arc are specially syntactically distinguishedsubclasses of the maximum class of studied objects, andthe concepts incidence of the sc-connector* and incidentof the incoming sc-arc* are treated as relations definedon the set of studied objects.
The family of all entered classes of studied objects(including the maximum class) is interpreted as Alphabetof SC-code. But, unlike other languages, the classes ofsyntactically distinguished elementary fragments of SC-code texts may overlap. For example, an sc-element canbelong to both the sc-element class and the sc-nodeclass, and can also belong to the sc-element class andsc-connector, and the sc-arc class, and the basic sc-arcclass.
This feature of the Alphabet SC-code makes it possibleto build syntactically correct sc-texts (texts of the SC-code) in the conditions of incompleteness of our initialknowledge about some sc-elements.
Lets consider the set-theoretic ontology of the SC-codesyntax:
sc-element<= subdividing*:• sc-node• sc-connector
sc-connector<= subdividing*:• sc-edge
34
• sc-arc
sc-arc⊃ base sc-arc
incidence of sc-connector*=> first domain*:
sc-connector=> second domain*:
sc-element⊃ incidence of incoming sc-arc*∈ binary relation∈ oriented relation∈ relation, elements of which there are no multisets
/*for binary relationships, this means no loops*/
incidence of incoming sc-arc*=> first domain*:
sc-arc=> second domain*:
sc-element
During the text processing, the following rules forclarifying their syntactic markup are executed:• if it has become known that sc-element, having a
sc-element label, is sc-node or sc-connector, then itis assigned a label sc-node or sc-connector, and thelabel sc-element is deleted;
• if it has become known that sc-element with thesc-connector label is sc-edge or sc-arc, then the sc-edge or sc-arc label is assigned, and the label sc-connector is deleted;
• if it has become known that sc-element with the sc-arc label is basic sc-arc, then the label basic sc-arcis assigned to it, and the label sc-arc is deleted.
Note some syntactic features of the SC-code.• The texts of the SC-code are abstract in the sense
that they abstract from the specific variant of theirencoding in the memory of the computer system.The coding of texts, in particular, depends on thevariant of the technical implementation of the mem-ory of a computer system. For example, the actualimplementation is the hardware implementation ofan associative non-linear memory in which thestructural reconfiguration of the stored informationis realized, in which information processing is re-duced not to a change in the state of memoryelements, but to a change in the configuration ofthe connections between them.
• The texts of the SC-code are structures of a graph-like type. All graph structures studied so far canbe easily represented in the SC-code (undirectedand oriented graphs, multigraphs, pseudographs,hypergraphs, networks, etc.). But, besides this, inthe SC-code, there are representable links betweenconnections, connections between whole structures
and much more. The SC-code is actually a graphlanguage, whose texts are graph-like structures.Thus, the graph theory with its appropriate exten-sion can become the basis for the description of thesyntax of the SC-code.
C. Semantic of SC-code
The simplicity of the SC-code syntax is determined bythe following semantic properties of sc-texts (characterconstructions belonging to the SC-code).• All (!) sc-elements, that is, elementary (atomic)
fragments of sc-texts, are signs (symbols) of variousdescribed entities. At the same time, each entitydescribed in the text SC-code must be representedby its sign;
• There are no signs other than sc-elements, sc-texts(i.e., there are no signs that include other signs);
• Any entity can be described by sc-text and, accord-ingly, is represented in this sc-text by its sign;
• All syntactically distinguished classes of sc-elements (i.e. all elements of Alphabet of SC-code)have a clear semantic interpretation – are classes ofsc-elements, each of which denotes an entity thatshares the same properties with all other entities,denoted by other sc-elements of the same class.
From the formal point of view, the denotational se-mantics of any sign construction (including sc-text) isa correspondence (more precisely, morphism) betweenthe set of all signs included in the sign constructionand the set of denotates of these signs (i.e. entities,denoted by these signs), as well as between the set of allsemantically significant (semantically interpreted) con-nections between the signs, and the set of correspondingconnections connecting either the denotations of all ofthe specified signs, or the denotations of some of thespecified signs directly with the rest of the signs fromthe mentioned signs themselves.
Consider the denotational semantics of sc-elementsbelonging to different syntactically distinguished classesof sc-elements, i.e. having different syntax labels.
If a sc-element is labeled as sc-element, then it candenote any described entity.
If sc-element has a label of sc-connector, which isincident to sc-element ei and to sc-element ej, then, onthe one hand, it a sign of pair ei, ej, and, on theother hand, is a model (reflection, description) of theconnection either between the denotate of sc-element eiand the denotate of sc-element ej, either between thedenotate of sc-element ei and the sc-element ej itself,either between the denotate of sc-element ej and sc-element ei itself.
If the sc-element has the label sc-node, then it denotesan entity that is not a pair.
If the sc-element has a label of sc-edge, which isincident to the sc-element ei and sc-element ej, then it
35
is, on the one hand, the undirected pair ei, ei, andon the other hand, is a model (reflection, description) ofthe connection either between the denotate of sc-elementei and the denotate of sc-element ej, either between thedenotate of sc-element ei and the sc-element ej itself,either between the denotate of sc-element ej and sc-element ei itself.
If sc-element has a label of sc-arc, which leaves sc-element ei and enters sc-element ej, then, on the onehand, it is a sign oriented pair <ei, ej>, and on theother hand, is a model (reflection, description) of theconnection either between the denotate of sc-element eiand the denotate of sc-element ej, either between thedenotate of sc-element ei and the sc-element ej itself,either between the denotate of sc-element ej and sc-element ei itself.
If sc-element has a label of base sc-arc, which outgoessc-element ei and enters sc-element ej, then, on the onehand, is a sign of oriented constant positive permanentpair of membership <ei, ej>, and, on the other hand, is amodel (reflection, description) of the connection betweenthe set, which is denoted by sc-element ei, and sc-elementej, which is one of the elements of the specified set.
We now turn to the consideration of the denotationalsemantics of the incidence of sc-connectors. Recall thateach sc-connector is semantically interpreted as a signof pair of sc-elements incident to this sc-connector.Accordingly, each pair of incidence of the sc-connector,not being a sc-element, is semantically interpreted as amodel (reflection, description) of the connection betweenthe pair of sc-elements, denoted by this sc- connector,and one of the two elements of this pair. At the sametime, the membership of the specified sc-element withinthe specified pair may have:
• constant or variable character depending on theconstancy or variability of the specified sc-connector;
• stationary (permanent) or non-stationary (situa-tional) character depending on the stationarity ornon-stationarity of the specified sc-connector.
The denotational semantics of the incidency of in-coming sc-arcs is given in a similar way. Each suchincidence pair is considered as a connection modelbetween the oriented pair, denoted by sc-arc and thesecond component of this pair (i.e. sc-element, in whichsc-arc ingoes). And similarly to the incidence of sc-connectors pairs incidence of incoming sc-arcs can haveconstant and variable character, as well as stationaryand non-stationary in depending on the nature of thecorresponding sc-arc.
The formal description of the denotational semanticsof SC-code by means of the SC-code itself is carriedout in the form of a hierarchical system of to-levelformal ontologies, presented in the form of SC-code. In
the knowledge base of Metasystems IMS.ostis all theseontologies are presented [16]. We list some of them.
Consider the Ontology of entities, within which thefollowing concepts are considered:
entity= sc-element<= subdividing*:• sc-constant• sc-variable
= sign of an arbitrary entity from a set ofpossible values
<= subdividing*:• stationary entity
= permanent entity• temporal entity
= non-stationary entity= time-varying entity⊃ temporary entity= temporarily existing entity
<= subdividing*:• material entity• terminal abstract entity• file
= primary (in perception) or final (in display)electronic image of the external informationstructure
• set= set of sc-elements<= subdividing*:• connection• structure• class<= subdividing*:• terminal entity class• relation
= class of connections• class of classes⊃ parameter
• class of structures
connection= tuple<= subdividing*:
36
• pair= binary connection⊃ sc-connector
/*some pairs of sc-elements in some periodsof time may not be syntactically designed asconnectors, but such a transformationnecessarily occurs*/
• non-binary connection
<= subdividing*:• non-oriented connection⊃ non-oriented pair
• oriented connection⊃ oriented pair
<= subdividing*:• constant connection
= (connection ∩ sc-constant)• variable connection
= (connection ∩ sc-variable)⊃ sc-variable values of which are constant
connections⊃ sc-variable values of which are variable
connections
pair= designation of a two-power set of sc-elements<= subdividing*:• non-oriented pair⊃ sc-edge
• oriented pair⊃ sc-arc⊃ pair of membership
<= subdividing*:• loop pair
= looped pair= pair, incident sc-elements of which coincide= couple being a multiset
• non-loop pair
pair of membership= connection describing the nature of the membership
of some sc-element in some set<= subdividing*:• pair of constant membership
= (pair of membership ∩ sc-constant)• pair of variable membership
= (pair of membership ∩ sc-variable)
<= subdividing*:• pair of permanent membership
= (pair of membership ∩ stationary entity)= pair of stationary membership
• pair of temporary membership= (pair of membership ∩ temporary entity)= pair of situational membership
<= subdividing*:• pair of positive membership
= pair of real membership• pair of fuzzy membership• pair of negative membership
= pair of nonexistent membership
⊃ pair of constant positive permanent membership= (pair of positive membership ∩ sc-constant ∩
stationary entity)⊃ base sc-arc
The following ontologies clarify (detail) the conceptsintroduced in Ontology of entitites.
The Ontology of sets clarifies the concept of setof sc-elements, considers various classes of sets (finite,infinite, countable, continual, multisets, sets without mul-tiple elements), different properties (characteristics) andrelations, given on sets (the power of sets, inclusion,union, subdividing, intersection, etc.).
The Ontology of relations deals with such conceptsas binary relation, unary relation, ternary relation, classof connections of equal power, class of connections ofdifferent power, arity of a relation, oriented, undirectedrelation, role relation, relation attributes*, relation do-main*, relation domain given attribute*, function, etc.
For the Ontology of relations, a lower level ontol-ogy is introduced – Ontology of binary relations andcorrespondences, which inherits all the properties ofrelations described in Ontology of relations, clarifies theconcept of binary relation and considers such concepts astransitive relation, symmetric relation, reflexive relation,equivalence relation, isomorphism, homomorphism, etc.
Next are introduced• Ontology of parameters and dimensions• Ontology of structures•• Ontology of subject domains•• Ontology of specifications•• Ontology of knowledge bases
• Ontology of variables and logical formulas• Ontology of temporal entities, which deals with
such concepts as non-stationary parameter (state),process, action, situation, sequence in time*, tem-poral decomposition* and other•• Ontology of actions
37
• Ontology of files and external information struc-tures
Some of the ontologies, presented in SC-code, have"general educational" character. This means that for qual-ity of mutual understanding between any subjects (bothusers and computer systems), i.e. for their qualitativesemantic compatibility, all these "general educational"ontologies, and in a coordinated, unified form, shouldbe known by all of subjects (!). Otherwise, there will beno mutual understanding.
The list of ontologies can be continued. All ontologiesare permanently changing (specified, improved). Themost important criterion of the quality of the hierarchicalsystem of ontologies is the stratification of methods forproblems solving corresponding to different ontologies –for each problem to be solved, it is desirable to aprioriknow within which ontology it can be solved.
It is obvious that, apart from "general education"ontologies, there is a large number of professional, spe-cialized ontologies, a consistent presentation and knowl-edge of which is necessary for mutual understanding(compatibility) of all those who work in the relevantprofessional field.
Thus, the denotational semantics of SC-code, likeany other language that claims to be universal, reflectsthe current state of our knowledge and, therefore, maychange. Obviously, these changes are most intense inspecialized and new areas of knowledge.
VII. REFINEMENT OF THE CONCEPT OF SEMANTICCOMPATIBILITY BASED ON THE STANDARD OF SENSE
INFORMATION REPRESENTATION
The most important stage in the evolution of anytechnology is the transition to the component designbased on the constantly updated library of reusablecomponents.
The main problems for the implementation of compo-nent design are• unification of components by form;• standards development to ensure compatibility of
these components.To implement component design of knowledge bases
the next is necessary:• universal language of knowledge representation;• universal procedure for the integration of knowl-
edge within the specified language;• development of a standard that provides semantic
compatibility of integrable knowledge (such a stan-dard is a consistent system of concepts used).
Even for the semantic representation of knowledge,a kind of semantic coordinates are needed, the role ofwhich is played by the used system of concepts (a kindof key signs), which, in turn, is described (specified,defined) by a hierarchical system of semantically inter-connected ontologies.
In other words, human knowledge must be brought to acommon "semantic denominator" (to a common semanticcoordinate system), which is the permanently refinedsystem of concepts specified as a unified ontology. Thisunified ontology is stratified to particular ontologies thatare sufficiently evolved independently from each other.
One of the criteria for the semantic compatibility ofnew information with the knowledge base into which thisinformation is immersed can be formulated as follows.
All signs that are new to the perceiving knowledgebase (in which these new signs are immersed) mustbe sufficiently specified (and defined for new concepts)through concepts known to the knowledge base.
The standard of sense representation of information(SC-code) makes it possible, on the one hand, to increasethe level of compatibility of computer systems, andon the other hand, to formally clarify the concept ofintegration of computer systems and their components.
Consider:• Semantic integration of two texts belonging to the
language of sense representation of information(SC-code). As a result of this integration, the twooriginal sc-texts are converted into one integratedtext;
• Semantic integration of two different models ofinformation processing, presented in the SC-code;
• A model of understanding the text of some externallanguage by translating the source external text intoan SC-code and then immersing the constructed sc-text into the knowledge base presented in the SC-code.
• Semantic integration of two computer systemsbased on the SC-code;
• Semantic compatibility of a computer system builton the basis of an SC-code with its users.
A. Refinement of the understanding process based on thesense presentation of information
It is obvious that the formalization of sense repre-sentation of information in the memory of a computersystem greatly simplifies clarifying how the process ofunderstanding new information takes place, which comesto the input of a computer system or is generated duringinformation processing. This process can be divided intothree stages:• translation of information from some external lan-
guage to an internal semantic language (SC-code).This stage is absent if new information is notentered from the outside, but is directly generatedin the memory of the computer system;
• immersion of new information presented as sc-textinto the current state of an information resourcestored in the memory of a computer system andalso represented as sc-text;
• alignment (agreement) of the concepts used in thenew externally entered or generated information
38
structure with the concepts used in the current stateof the information resource stored in the computersystem’s memory.
Consider each of these steps in more detail.
Translation of information from some external lan-guage into the SC-code is simplified due to the fact that:• means of SC-code allow to describe syntax of
external language, because the universality of theSC-code allows, with its help and with any degreeof detail, to describe any objects, including suchcomplex systems of the external environment ofcomputer systems as external languages;
• the process of the semantic analysis of the sourcetext of an external language can be performed bymanipulating the texts of the SC-code and, as aresult, obtaining a description of the structure ofthe source text that has sufficient completeness(detailing) for the subsequent generation of a textthat is semantically equivalent to it;
• SC-code can be used to describe semantics of anexternal language, treating it as a description of theproperties of morphisms between sc-texts describ-ing the syntactic structure of the source externaltexts, and sc-texts that are semantically equivalentto these source texts;
• the process of generating sc-text, semanticallyequivalent to the original external text, can alsobe performed by sc-texts manipulating.
Thus, the effectiveness of using of SC-code for trans-lating text from some external language into SC-code isdue to the fact that using the SC-code we can describeboth the syntax and semantics of an external language.We can parse the external text and the subsequent gen-eration of sc-text, semantically equivalent to the originalexternal text, while remaining within the SC-code.
Immersion (integration) of a new generated sc-textinto a given sc-text (for example, into the knowledgebase presented in SC-code) reduces to merging (identi-fication) of some sc-elements of a new sc-text with the sc-elements that are part of the given sc-text. Thus, the taskof immersing a new sc-text into a given sc-text reducesto the task of constructing a set of pairs of synonymoussc-elements, one of which is part of the new submersiblesc-text, and the second is the part of the given sc-text.
The establishment of pairs of synonymous sc-elementsis carried out:• by searching for pairs of sc-elements that have
agreed external names that match (we emphasizethat all used concepts must have correspondingmatching external names);
• by logical reasoning, using logical formulas of thefollowing types:•• non-existence formulas;•• formulas of existence and uniqueness;
•• formulas for the existence of a finite and in-dicated number of values of the correspondingvariables.
To simplify the establishment of pairs of synonymoussc-elements, some statements about non-existence, exis-tence and uniqueness, existence of a given finite numberof structures of a given type can be reformulated in amore "constructive" key with the explicit introduction ofthe synonymy of sc-elements relation. So, for example,instead of the statement that “For each pair of points,there is a single straight line passing through them”, thefollowing wording can be used: “If lines pi and pj passthrough the points ti and tj, then either pi = pj, or ti =tj, or ti /∈ pi, or tj /∈ pi, or ti /∈ pj, or tj /∈ pj”.
A sufficiently detailed description of the example ofsc-text immersion in the knowledge base, also presentedin SC-code, is given in Section IX of the article [1] –Example 4.
Alignment of concepts, used in the new integrable(introduced, immersed) sc-text, with the concepts usedin the given integrating sc-text, is as follows:• The specified integrating sc-text (usually this is
the knowledge base presented in SC-code) mustexplicitly contain:•• information about the current status (state, char-
acter) of using of each concept known toknowledge base and used either directly in theknowledge base itself or by external actors,information from which can be input to thespecified knowledge base;
•• information about the current status (state, char-acter) of use of each external sign (most oftena term, name) corresponding to each conceptused, as well as some well-known entities thatare not concepts;
• Integrable (input, immersible) text must:•• use agreed concepts and the corresponding
agreed external signs (terms, names) as muchas possible;
•• include definitions of all concepts that are new,unknown in the integrating text (the definitionshould use only those concepts that are knownto the integrating text);
• To solve the problem of the used concepts align-ment for the current state of the knowledge baseand for the new text (integrated) into this textknowledge base, all concepts used in the knowledgebase are divided into:•• currently agreed (recognized) and not changing
their status;•• obsolete = concepts used before or rarely used
now;•• obsoleting = concepts for which, for a given
period of time, their status is replaced from the39
status of the agreed concept to the status of theobsolete concept;
•• returning = concepts, the status of whichchanges from the status of the obsolete conceptto the status of the agreed concept;
•• proposed new concepts = new concepts under-going approval = concepts, the status of whichchanges from the status of proposed to thestatus of either approved or obsolete = agreeingconcepts;
•• approved concepts = concepts that have beensuccessfully negotiated;
•• rejected concepts = concepts whose agreementresults are negative;
•• introduced new concepts = concept, the statusof which changes from status of the approvedconcept to the status of a agreed concept =concepts introduced into use.
Thus, the process of alignment of concepts, the goal ofwhich is to reduce all the concepts used in the integratedsc-text, to the agreed concepts of knowledge base, iscarried out under the conditions of a permanentchange in the status of the used concepts and constantincrease numbers of such concepts.
It should be distinguished:• family of all concepts known to knowledge base at
the current moment;• the current status of all these concepts;• the set of all transition processes aimed at changing
the status of concepts and being implemented at themoment.
Note also that the permanent process of agreement ofall the concepts used is a necessary condition for en-suring compatibility (integrability) of SC-code texts. Butto ensure compatibility of SC-code texts, a permanentprocess is needed to agree not only the concepts usedthemselves, but also the corresponding external signs(names, terms). Moreover, external signs (names) andtheir agreement may be required not only for concepts,but also for entities of other types (for example, for peo-ple, settlements, geographical objects, historical events,etc.).
We emphasize at the same time that the principlesof organizing the agreement of external signs (names)are similar to the principles of organizing the agreementof concepts discussed above in the context of theirpermanent change. So, for example, each connection ofthe be external sign* relation. linking the sc-sign of someentity with the sc-node denoting the external file of signof the specified entity, as well as each concept, can beput in compliance with its current status (agreed, obso-lete, obsoleting, returning, proposed, approved, rejected,included).
Finishing the consideration of the model of under-standing as a model of semantic input of some text, not
necessarily belonging to SC-code, into the given text SC-code, we make several remarks.
Understanding may be distorted (including contradic-tory) and superficial (incomplete) due to poor-qualityimmersion of new information in the current state of aninformation resource stored in the memory of a computersystem (error in identifying signs and, as a consequence,incorrectly established synonymy, or incompleteness ofidentification, not all new signs, synonymous with theknowledge base, are merged with their synonyms).
The problem of understanding, mutual understand-ing between people, between computer systems, betweencomputer systems and their users is the epicenter ofthe modern stage of evolution of computer systems andis waiting to be solved. The deeper we penetrate theformalization of the process of understanding (especiallythe understanding of the texts of natural language), themore and more it is surprising that people still somehowunderstand each other, although not always. More often itis not an understanding, but an illusion of understanding.Here it is appropriate to recall the well-known phrase:“Happiness is when you are understood.”
B. Unification and compatibility of various models ofproblem solving
Our proposed approach to a significant increase in thelevel of compatibility (integrability) of various problemsolving models is as follows:• All information stored in the memory of each prob-
lem solver (both the actual information processedand the interpreted skills stored in the memory, forexample, a different type of program), is presentedin the form of a sense representation of this infor-mation (in SC-code);
• Actually, the solution of each task is carried outby a team of agents working on a common sense(semantic) memory and interpreting the skills storedin the same memory (these agents will be called sc-agents);
• The integration of two different models for prob-lems solving is reduced:•• to combining the memory of the first model
with the memory of the second model;•• to the integration of all sc-text stored in the
memory of the first model, with sc-text storedin the memory of the second model (this inte-gration is carried out by mutual immersion ofthese sc-texts into each other, i.e., by mergingtogether synonyms, as well as by aligning theconcepts they use);
•• to the union of the set of agents included in thefirst model with the set of agents included inthe second model of problem solving.
Thus, the unification of problem solving models byreducing these models to the form of sc-models (i.e.,
40
sc-text processing models) improves the compatibilitylevel of these models due to the transparent integrationof processed and integrable sc-texts and the trivial unionprocedure for sets of sc-agents. The simplicity of theprocedure for union of sets of sc-agents correspondingto different models of problem solving is due to the factthat there is no direct interaction between these agents,and the initiation of each of them is determined by sc-agent itself, as well as the current state of informationstored in memory.
Thus, as a basis for the unification of informationprocessing principles in computer systems, it is proposedto use the multi-agent approach. The focus on a multi-agent approach is due to the following main advantagesof this approach [5]:
• autonomy (independence) of agents, which allowsto localize changes made to the system during itsevolution, and reduce the corresponding labor costs;
• processing decentralization, i.e. the absence of asingle monitoring center, which also allows to lo-calize changes made to the system.
But the modern principles of building multi-agent sys-tems when applied to multi-agent processing of knowl-edge bases have several disadvantages:
• agent knowledge is represented using highly spe-cialized languages, often not intended to representknowledge in a broad sense and ontologies inparticular;
• most modern multi-agent systems assume thatagents interact by exchanging messages directlyfrom the agent to the agent;
• the logical level of interaction between agents isrigidly tied to the physical level of the implemen-tation of a multi-agent system;
• the environment with which agents interact, is spec-ified separately by the developer for each multi-agent system, which leads to significant overheadand incompatibility of such multi-agent systems.
It is proposed to eliminate the listed disadvantages byusing the following principles:
• agents are proposed to be communicated by specify-ing (in the common memory of a computer system)actions (processes) performed by agents and aimedat problems solving;
• the external environment for agents is the samecommon memory;
• the specification of each agent is described bymeans of a knowledge representation language inthe same memory;
• synchronization of agents’ activities is proposed atthe level of the processes they perform
• each information process at any time has associativeaccess to the necessary fragments of the knowledgebase stored in common memory.
C. Semantic compatibility of computer systems
The compatibility level of computer systems is de-termined by the laboriousness of the implementation ofintegration procedures (integration, connection of knowl-edge of these systems), as well as the laboriousness anddepth of integration of these systems problem solvers(skills and interpreters of these skills). We emphasize atthe same time that the integration can be different – fromeclecticism to hybridity and synergy, the distance is ofenormous size.
Compatible computer systems are computer systemsfor which there is an automatically performed integrationprocedure that turns these systems into a single hybridsystem, within the framework of which each originalcomputer system can free to use any necessary knowl-edge and skills that are part of another source computersystem.
The integral computer system can be considered as aproblem solver, integrating several models of problemsolving and having the means of interaction with theexternal environment (with other computer systems, withusers).
Thus, in order to increase the compatibility level ofcomputer systems, it is necessary to convert them to theform multi-agent systems, working on a common seman-tic memory, in which the information is represented bytexts of SC-code. Such unified computer systems it isnot always advisable to directly integrate (integrate) intolarger computer systems. Sometimes it is more expedientto combine them into teams of interacting computersystems. But when creating such groups of computersystems, the unification and compatibility of such sys-tems are also very important, since significantly simplifythe provision of a high level of mutual understanding.For example, contradictions between computer systemsbelonging to a team can be detected by analyzing theconsistency of virtual unified knowledge base of thisteam. Moreover, the consistency of the specified virtualknowledge base can be considered one of the criteria forsemantic compatibility of the systems included in therelevant team.
D. Advantages of the semantic presentation of informa-tion
Why is it appropriate to move to the semantic rep-resentation of information in the memory of computersystem:
• sense representation of information is an objective,independent of subjectivity and diversity of syntac-tic decisions, way of information representation;
• within the framework of the semantic presentation,the procedure of integrating knowledge and im-mersing new knowledge into the knowledge baseis greatly simplified;
41
• greatly simplifies the procedure for bringing a dif-ferent type of knowledge to a general form (to anagreed system of concepts used);
• greatly simplifies the process of integrating variousproblem solvers and whole computer systems;
• significantly simplifies the automation of the perma-nent process of supporting semantic compatibility(consistency of concepts and ontologies) for com-puter systems in the context of their continuousimprovement;
• based on the proposed standard of sense repre-sentation of information significantly simplifies theintegration of various disciplines in the field ofartificial intelligence, i.e. building a general formaltheory of intelligent systems, since building a gen-eral formal model of intelligent systems requires abasic language, within which one could easily movefrom information (from knowledge) to metainfor-mation (to metaknowledge, to specifications ofinitial knowledge). This is confirmed by the factthat:•• the overwhelming number of concepts of artifi-
cial intelligence has a metalinguistic character;•• SC-code represents the unity of the language
and the metalanguage, remaining within theframework of a simple syntax;
•• the formal semantic refinement of almost everyconcept of artificial intelligence requires a priorformal refinement of the corresponding objectlanguage. So, for example, how can one speakstrictly about the language of ontologies (i.e„the language of the specification of subjectdomains) without specifying the language ofrepresentation of these subject domains them-selves. How can one speak strictly about thelanguage of the description of information pro-cessing methods without specifying the lan-guage of the representation of this processedinformation itself.
VIII. SEMANTIC COMPUTER SYSTEMS ANDTECHNOLOGIES
We propose a solution to the problems of moderninformation technologies by moving to the sense rep-resentation of information in the memory of computersystems actually transforms modern computer systems(including modern intelligent systems) into semanticcomputer systems, which, consequently, are not an alter-native branch of development of computer systems, but anatural stage of their evolution, aimed at ensuring a highlevel of their learnability and, first of all, compatibility.
The architecture of semantic computer systems (seefig. 1) almost coincides with the architecture of intelli-gent systems based on knowledge bases. The differencehere is that in the semantic computer systems:
• the knowledge base has sense representation;• the knowledge and skills interpreter is a group of
agents processing knowledge base.
Figure 1. Architecture of ostis-system
As a consequence, semantic computer systems have ahigh level of learnability, i.e. ability to quickly acquirenew and improve already acquired knowledge and skillsand at the same time not have any restrictions on thetype of acquired and improved knowledge and skills, aswell as on their sharing.
Moreover, with the agreement of relevant standards,as well as with the permanent improvement of thesestandards and with their competent support in the condi-tions of intensive evolution of both the standards them-selves and semantic computer systems (this is about thepermanent support of the correspondence between thecurrent state of computer systems and current state ofevolving standards), semantic computer systems and theircomponents have a very high degree of compatibility.
This, in turn, virtually eliminates the duplication ofengineering solutions and makes it possible to signifi-cantly speed up the development of semantic computersystems using a constantly expanding library of reusableand compatible components.
The main leitmotif of the transition from moderncomputer systems (including intelligent) to semanticcomputer systems, i.e. computer systems based on thesense representation of all the information stored in itsmemory is the creation of general semantic theory ofcomputer systems, which includes:
• semantic theory of knowledge and knowledgebases;
• semantic theory of problems and models for solvingthem;
42
• semantic theory of interaction of information pro-cesses;
• semantic theory of user and, including natural lan-guage interfaces;
• semantic theory of non-verbal sensory-effector in-terfaces;
• theory of universal interpreters of semantic modelsof computer systems and, in particular, the theoryof semantic computers.
The epicenter of the next stage of information tech-nology development is the solution to the problem ofproviding semantic compatibility of computer systemsand their components. To solve this problem is needed• transition from traditional computer systems and
from modern intelligent systems to semantic com-puter systems;
• standard of development of semantic computer sys-tems.
Obviously, semantic computer systems are the newgeneration of computer systems that eliminate many ofthe shortcomings of modern computer systems. But forthe mass development of such systems, an appropriatetechnology is needed, which should include• theory of semantic computer systems and a complex
of all standards ensuring compatibility of developedsystems;
• methods and design tools for semantic computersystems;
• methods and tools of permanent improvement of thetechnology itself.
Our proposed technology for developing semanticcomputer systems is named OSTIS (Open SemanticTechnology for Intelligent Systems).
The basis of this technology is SC-code - the standardof sense representation of information in the memory ofcomputer systems developed by us.
Overall, OSTIS Technology is• standard for semantic computer systems, ensuring
the semantic compatibility of systems conformingto this standard;
• methods of construction of such computer systemsand their improvement in the course of their oper-ation;
• tools and means for building and improving thesesystems•• language means;•• library of typical technical solutions;•• tools• • • for synthesis and modification;• • • for analyzing, verifying, diagnosing, test-
ing;• • • for eliminating detected errors and flaws.
It is essential to emphasize that OSTIS Technology isnot just standard of semantic computer systems, but
a standard that is constantly and intensively improvedduring the continuous expansion and improvement ofthe formalization of the types of knowledge used andmodels for solving problems by reaching a consensus(coordination of points of view) with the participation ofall interested individuals and legal entities.
The principal thing is that OSTIS Technology allowsto create systems that do not necessarily have to solveintelligent tasks, but this implementation of computersystems provides:• compatibility;• high degree of flexibility, which allows unlimited
expansion of the functionality of computer systems,including the ability to solve intelligent tasks.
We list the principles underlying OSTIS Technology:• orientation to the semantic unambiguous representa-
tion of knowledge in the form of semantic networksthat have a basic set-theoretic interpretation, whichprovides a solution to the problem of the diversityof the forms of representation of the same meaning,and the problem of ambiguity of semantic interpre-tation of information structures;
• use of an associative graph-dynamic memorymodel;
• application of agent-based knowledge processingmodel;
• implementation of OSTIS Technology in the formof intelligent IMS.ostis Metasystem, which itselfis built on OSTIS Technology and supports thedesign of computer systems developed by OSTISTechnology;
• ensuring a high level of flexibility, stratification,reflexivity, hybridity, compatibility, and, as a result,learnability of designed systems.
The advantages of OSTIS Technology include:• OSTIS Technology has an open character both for
its users (developers of applied intelligent systems)and for those who wish to participate in its improve-ment;
• OSTIS Technology is focused on a constant increasein the pace of its evolution;
• OSTIS Technology is the basis for solving the prob-lems of semantic compatibility of various scientificand technical knowledge, since it is focused on theformalization of interdisciplinary connections of themost diverse type.
Perspective directions of OSTIS Technology applica-tion are:• Development on the basis of OSTIS Technology
of a particular technology of designing intelligentreference systems, intelligent semantic textbooks,learning systems and intelligent help-systems invarious fields;
• A complete set of compatible semantic electronictextbooks across the entire set of school subjects;
43
• Intelligent personal assistants (secretaries, refer-ents), providing personalized information services,integration of available services, monitoring andcontrol of users;
• Intelligent control systems of various enterprises,organizations, projects based on ontologies and aformal description of actions performed, events,situations;
• Intelligent automation systems for designing vari-ous classes of artificial systems based on ontologicalmodels;
• Portals of scientific knowledge and semantic toolsof supporting the development of various scientificand technical areas;
• Distributed global semantic knowledge space,which is the result of integrating the knowledgebases of all systems built on OSTIS Technology andinterconnected by a global network;
• Intelligent systems of excursion service;• Intelligent systems of complex individual medical
monitoring and service;• Intelligent robotic systems;• Smart living environment (smart home, smart road,
smart city).
IX. ECOSYSTEM OSTIS
Ecosystem OSTIS= Sociotechnical ecosystem, which is a group of
interacting semantic computer systems and providespermanent support for the evolution andcompatibility of all its member systems, throughouttheir life cycle.
= Unlimitedly expandable team of constantly evolvingsemantic computer systems that interact with eachother and with users to solve complex problems in acorporate way and to constantly maintain a highlevel of compatibility and mutual understanding ininteraction both with each other and with users
Since the above-considered OSTIS Technology is fo-cused on the development of semantic computer systemswith a high level of learnability and, in particular, a highlevel of semantic compatibility, and since learnability andcompatibility are only ability to learn (i.e., to high ratesof expansion and improvement of their knowledge andskills), as well as ability to ensure a high level of mutualunderstanding (coherence), some kind of environment,social engineering infrastructure, is needed in the frame-work of which most comfortable conditions have beencreated for the implementation of the above abilities. Thisenvironment is named by us Ecosystem OSTIS, whichis a group of interacting (via the Internet):
• semantic computer systems, built according to stan-dard of OSTIS Technology (such systems will becalled ostis-systems);
• users of the specified ostis-systems (both end usersand developers);
• some computer systems that are not ostis-systems,but they are considered as additional informationresources or services.
A. Compatibility support between computer systems ofEcosystem OSTIS
The main purpose of Ecosystem OSTIS is to en-sure compatibility of computer systems included in theEcosystem OSTIS both at the stage of their develop-ment and during their operation. The problem here isthat during the operation of the systems included inthe Ecosystem OSTIS, they may change due to whichcompatibility may be violated.
The tasks Ecosystem OSTIS are:• operative implementation of all agreed changes to
the ostis-systems standard (including changes to thesystems of the used concepts and their correspond-ing terms);
• permanent support of a high level of mutual under-standing of all the systems included in the Ecosys-tem OSTIS, and all their users;
• corporate solution of various complex problemsrequiring coordination of activities of several (mostoften, a priori unknown) ostis-systems, as well as,possibly, some users.
The Ecosystem OSTIS is a transition from independent(autonomous, separate, integral) ostis-systems to collec-tives of independent ostis-system m, i.e. to distributedostis-systems. The following types of ostis-systems canbe distinguished by the hierarchy level:• atomic embedded ostis-system
= ostis-system integrated into the independentostis-system, but not into the composition ofanother embedded ostis-system
• non-atomic embedded ostis-system= ostis-system, which is integrated into the
independent ostis-system, and includes someother embedded ostis-systems
⊃ user interface
• independent ostis-system= consistent ostis-system, which must
independently perform the corresponding set oftasks and, in particular, interact with theexternal environment (verbally – with usersand other computer systems, and non-verbally)
• collective of ostis-systems= a group of communicating ostis-systems, which
can include not only independent ostis-systems,but also collectives of ostis-systems
= distributed ostis-sysstem
• Ecosystem OSTIS∈ maximum collective of ostis-systems
44
∈ collective of ostis-systems that is not part ofanother collective of ostis-systems
We emphasize that the independent ostis-systems,which are part of the Ecosystem OSTIS, are met specialrequirements:
• they must have all the necessary knowledge andskills for messaging and purposeful organizationof interaction with other ostis-systems belonging toEcosystem OSTIS;
• under the conditions of constant change and evo-lution of the ostis-systems included in the Ecosys-tem OSTIS, each of them should itself monitorits compatibility (consistency) with all the othersostis- systems i.e. should independently maintainthis compatibility, coordinating with other ostis-systems all changes that need to be coordinated,occurring in themselves and in other systems.
• Each system included in the Ecosystem OSTIS must:•• study intensively, actively and purposefully
(both with the help of teachers and developers,and independently);
•• inform all other systems about proposed orfinally approved changes in ontologies and, inparticular, in the set of concepts used;
•• accept from other ostis-systems proposals forchanges in ontologies (including the set of con-cepts used) for agreement or approval of theseproposals;
•• implement approved changes in ontologiesstored in its knowledge base;
•• help to maintain a high level of semantic com-patibility not only with other ostis-systems in-cluded in Ecosystem OSTIS, but also with itsusers (i.e. to train them, inform them aboutontology changes).
The Ecosystem OSTIS is a form of realization, im-provement and application of OSTIS Technology and,therefore, is a form of creation, development, self-organization of the market for semantically compatiblecomputer systems and includes all the necessary re-sources for this – personnel, organizational, infrastruc-tural.
The Ecosystem OSTIS is mapped to its integratedknowledge base, which is virtual union of knowledgebases of all ostis-systems included in Ecosystem OSTIS.The quality of this knowledge base (completeness, con-sistency, clearness) is a permanent attention of all theindependent ostis-systems included in Ecosystem OSTIS.Accordingly, each specified ostis-system is associatedwith its own knowledge base and its own hierarchicalsystem of sc-agents.
By purpose, the ostis-systems included in the Ecosys-tem OSTIS can be:
• assistants to specific users or specific user teams;
• standard embedded subsystems of ostis-systems;• information and tool support systems for designing
various components and various classes of ostis-systems;
• information and tool support systems for designingor producing various classes of technical and otherartificially created systems;
• knowledge portals for various scientific disciplines;• automation systems for managing various complex
objects (industrial enterprises, educational institu-tions, departments of universities, specific students);
• intelligent reference and help-systems;• intelligent learning systems, semantic electronic tu-
torials;• intelligent robotic systems.
B. Compatibility support between computer systems andtheir users in the Ecosystem OSTIS
There are two aspects to maintaining compatibility andunderstanding in the Ecosystem OSTIS• compatibility support between ostis-systems in-
cluded in Ecosystem OSTIS;• compatibility and mutual understanding between
the ostis-systems included in the Ecosystem OSTISand their users, with active encouragement from theEcosystem OSTIS, so that each user of EcosystemOSTIS at the same time is not only its active enduser, but also its active developer.
Thus, to ensure high operational efficiency and highrates of evolution of Ecosystem OSTIS, it is necessary toconstantly increase the level of information compatibility(level of mutual understanding) not only between thecomputer systems that make up the Ecosystem OSTIS,but also between these systems and their users. One ofthe ways to ensure such compatibility is the desire toensure that each user’s knowledge base (picture of theworld) becomes a part (fragment) Joint Knowledge Baseof Ecosystem OSTIS. This means that each user shouldknow how the structure of each scientific and technicaldiscipline is arranged (objects of research, subjects ofresearch, definitions, statements, etc.), and how differentdisciplines can be interconnected.
The formation of such system building skills of thepicture of the World should be started from the secondaryschool. For this purpose, it is necessary to create aset of compatible intelligent learning systems for allsecondary education disciplines with clearly describedinterdisciplinary connections [17], [18]. Thanks to this,it is possible to prevent the users from forming the"mosaic" picture of the World as a multitude of poorlyrelated disciplines. And this, in turn, means a signif-icant improvement in the quality of education, whichis absolutely necessary for high-quality operation ofnext-generation computer systems – semantic computersystems.
45
Users and, first of all, the developers of EcosystemOSTIS should have a high level of:• mathematical culture (formalization culture) when
building a formal model of the environment inwhich an intelligent system functions, formal mod-els of the problems it solves and formal models ofvarious methods of problems solving it uses;
• system culture, which allows to adequately assessthe quality of the developed systems from the pointof view of the general theory of systems and, inparticular, assess the overall level of automationimplemented with the help of these systems. Systemculture involves the desire and ability to avoid eclec-ticism, the desire and ability to provide high-qualitystratification, flexibility, reflexivity, as well as high-quality maintenance, a high level of learnability anda comfortable user interface of the systems beingdeveloped;
• technological culture, ensuring compatibility of thedeveloped systems and their components, as well asthe continuous expansion of the library of reusablecomponents of the created systems and assuming ahigh level of design discipline;
• ability to work in a team of developers of high-tech systems, which implies a high level of abilityto work at interdisciplinary junctions, a high levelof communication skills and agreeability, i.e. theability not only to defend one’s point of view, butto coordinate it with the views of other developersin the interests of development Ecosystem OSTIS;
• activity and responsibility for the overall result –high rates of evolution Ecosystem OSTIS in general.
Thus, the high evolution rates of Ecosystem OS-TIS are provided not only by the professional qual-ifications of users (knowledge of OSTIS Technology,current status and problems of Ecosystem OSTIS andskills of using OSTIS Technology and intelligent sys-tems included in the Ecosystem OSTIS), but also therelevant human qualities. Obviously, the modern levelof agreeability, activity and responsibility cannot be thebasis for the evolution of such systems as EcosystemOSTIS.
Support compatibility of Ecosystem OSTIS with itsusers carried out as follows:• each ostis-system includes embedded ostis-systems
oriented on•• permanent monitoring of the activities of end
users and developers of this ostis-system,•• analysis of the quality and, above all, the cor-
rectness of this activity,•• permanent unobtrusive personalized training
aimed at improving the quality of user activity,i.e. to improve their skills;
• within the Ecosystem OSTIS there are ostis-systems,specifically designed to train users of Ecosystem
OSTIS to the basic recognized knowledge and skillsto perform the corresponding classes of tasks. Thisincludes the knowledge corresponding to the levelof secondary education, and knowledge correspond-ing to the basic disciplines of higher education inthe field of informatics (and, in particular, in thefield of artificial intelligence), and basic knowledgeof OSTIS Technology and about Ecosystem OSTIS.
The problem of creating a market for compatiblecomputer systems is the challenge to modern scienceand technology. Scientists working in the field of arti-ficial intelligence require the ability to work collectivelyon solving interdisciplinary problems and bring thesesolutions to a general integrated theory of intelligentsystems, involving the integration of all areas of ar-tificial intelligence, and to technologies available to awide range of engineers. Intelligent systems engineersare required to actively participate in the developmentof relevant technologies and to significantly increasethe level of mathematical, systemic, technological, andorganizational-psychological culture.
But the main task here is to reduce the barrier betweenscientific research in the field of artificial intelligence andengineering in the development of intelligent systems.For this, science should be constructive and focused onthe integration of its results in the form of an integratedtechnology for developing intelligent systems, and engi-neering, having realized the knowledge-intensiveness ofits activities, should actively participate in the develop-ment of technologies.
Particular emphasis in the Ecosystem OSTIS is placedon the ongoing process of agreement of ontologies (and,first of all, on the harmonization of the family of allused concepts and terms corresponding to these concepts)between all (!) active subjects of Ecosystem OSTIS –between all ostis-systems and all users.
In the presence of ostis-systems, which are personalassistants of users in cooperation with the EcosystemOSTIS, this whole Ecosystem will be perceived by usersas a single intelligent system uniting all informationresources and services available in the Ecosystem OSTIS.
The principles of organization of Ecosystem OSTIScreate all the necessary conditions for attracting scien-tific, organizational and financial resources to the devel-opment and improvement of OSTIS Technology, whichwill be aimed at developing methods and means of arti-ficial intelligence and forming a market for semanticallycompatible intelligent systems.
X. IMS.OSTIS METASYSTEM
The effectiveness of any technology, including OSTISTechnology [16] is determined not only by the time termsfor the creation of artificial systems of the correspondingclass, but also by the rates of improvement of thetechnology itself (rates of improvement of automationtools underlying technology).
46
To fixate the current state of OSTIS Technology, aswell as to organize its effective use and its permanentimprovement with the participation of scientists workingin the field of artificial intelligence, and engineers whodevelop semantic computer systems for various purposesinto the OSTIS Ecosystem the IMS.ostis [16] is intro-duced, the purpose of which makes it key ostis-systemwithin the OSTIS Ecosystem.
IMS.ostis Metasystem= Intelligent metasystem of integrated informational
and instrumental support for the design ofcompatible semantic computer systems, which is aform of realization of the general theory andtechnology of designing semantic computer systemsand which maintains a high rate of evolution of thistheory and technology
= Intelligent MetaSystem for intelligent systems design= IMS.ostis= Intelligent System Framework= Intelligent metasystem of complex support for the
design of compatible semantic computer systemsusing OSTIS Technology
= Framework of ostis-systems= Framework IMS.ostis
The IMS.ostis Metasystem is in the Ecosystem OSTIS akey intelligent system that supports not only the design ofnew intelligent systems and not only the replacement ofobsolete components in the intelligent systems includedin the Ecosystem OSTIS, but also inclusion (integration)in the Ecosystem OSTIS of newly created intelligentsystems.
IMS.ostis Metasystem is focused on the developmentand practical implementation of methods and tools com-ponent design and semantically compatible intelligentsystems, which provides the ability to quickly createintelligent applications for various purposes.
The areas of practical application of the componentdesign technology of semantically compatible intelligentsystems are not limited by anything.
A. Structure of developed ostis-systems
The architecture of computer systems developed byOSTIS Technology is clearly stratified into two subsys-tems:• knowledge base, which is a complete semantic
model of an intelligent system (which will be calledthe sc-model of the intelligent system or the sc-model of the knowledge base of the intelligentsystem, as it is formed as a coherent sign constructbelonging to SC-code – the base language of the in-ternal sense representation of knowledge in memoryof ostis-systems);
• basic universal interpreter of the semantic modelof an intelligent system stored in its memory (the
interpreter of the sc-model of the knowledge baseof an intelligent system).
These subsystems of ostis-systems can be developedcompletely independently of each other with the obser-vance of clear requirements imposed by OSTIS Tech-nology which consist in the interpretation of the syntaxand semantics of SC-code that are identical for thesesubsystems which is the universal language of the inter-nal semantic representation of knowledge in the memoryostis-systems, as well as the syntax and semantics of SCPlanguage (Semantic Code Programming), which is thesublanguage of SC-code and is a basic language of agent-oriented programming which is focused on processing ofsign structures, belonging to SC-code.
The considered stratification of ostis-systems to com-patible with each other knowledge base and the knowl-edge base interpreter, firstly, provides ample opportuni-ties for a wide variety of implementation options for theinterpreter of sc-models of knowledge bases (includingvarious implementations of semantic computers with as-sociative graph-dynamic, reconstructable memory) and,secondly, makes it possible to easily transfer (reload)the knowledge base of an intelligent system into thememory of another knowledge base interpreter. Thesecond possibility means the platform independence ofthe intelligent systems developed by OSTIS Technology,since the various implementations of interpreters of sc-models of knowledge bases are nothing but differentplatform options for ostis-systems implementing.
Thus, if there is a sufficiently effective version ofthe implementation of the interpreter of sc-models ofknowledge bases, the development of ostis-system comesdown to designing sc-model of its knowledge base [15],which includes itself:• sc-model of the integrated problem solver of this
ostis-system [19], which, in turn, includes:•• sc-models of classes of problems to be solved
(in particular, stored programs of high-levellanguages);
•• scp-programs of knowledge processing agents;• sc-model of the integrated interface of the ostis-
system, which is a built-in ostis-system, focused onsolving interface problems related to ensuring thedirect interaction of the ostis-system with the exter-nal environment (both non-verbal receptor-effectorinteraction, and verbal interaction with users, withother ostis-systems, with other computer systems).
B. Technical implementation of the IMS.ostis Metasys-tem
The purpose of IMS.ostis Metasystem is the implemen-tation of the design technology of semantically compat-ible computer systems in the form of a metasystem builtusing the same technology and providing comprehensiveinformation and tool support for designing semantically
47
compatible computer systems. The composition of thespecified metasystem includes:
• full description of the technology itself;• history of the evolution of technology;• description of technology usage rules;• description of the organizational infrastructure
aimed at the development of technology;• library of reusable compatible components of intel-
ligent systems;• methods and tools for designing various types of
intelligent system components;• technical tools of coordinating the activities of
project participants, aimed at the continuous im-provement of technology.
Tasks of IMS.ostis Project are:
• To develop IMS.ostis Metasystem, which providesfast component design of semantically compatiblecomputer systems for various purposes.
• To develop methods and tools to ensure the inten-sive development of the market for semanticallycompatible applied intelligent systems created onthe basis of IMS.ostis Metasystem.
• To develop methods and means to stimulate theintensive development of the IMS.ostis Metasystem.
The scientific novelty of IMS.ostis Metasystem is theunification of the representation of various types ofinformation in the memory of computer systems basedon the sense (semantic) presentation of this information,which ensures:
• avoiding duplication of the same information indifferent intelligent systems and in different com-ponents of the same system;
• semantic compatibility of various components ofintelligent systems and various intelligent systemsin general;
• is a significant expansion of libraries of compatiblereusable components of computer systems due to"large" components and, in particular, typical sub-systems.
The principles of the technical implementation ofIMS.ostis Metasystem completely coincide with the prin-ciples of the technical implementation of applied intelli-gent systems developed with the help of this metasystem.Thus, the IMS.ostis Metasystem is an intelligent systemdesigned for comprehensive information and tool supportfor designing semantically compatible computer systems,the purpose of which is not imposed any restrictions.
The knowledge base of IMS.ostis Metasystem includes:
• current state of models and methods used in thedevelopment of intelligent systems using IMS.ostisMetasystem;
• systematic library of reusable and compatible com-ponents of intelligent systems;
• description of design tools for various types of in-telligent systems components (fragments of knowl-edge bases, problem solvers, user interfaces);
• description of the tools of coordinating collectiveactivities aimed at the continuous development ofIMS.ostis Metasystem;
• description of the evolution history of IMS.ostisMetasystem;
• description of design tools for various classes ofintelligent systems.
The problem solver and the user interface of IMS.ostisMetasystem provide support for the entire complex of de-sign tasks solved by the developers of applied intelligentsystems, as well as by the developers of the IMS.ostisMetasystem.
The IMS.ostis Project is implemented in the form ofinteraction of IMS.ostis Metasystem with its users and isbased on the following principles:
• In order to stimulate the development of the marketof compatible application intelligence systems de-veloped with the help of IMS.ostis Metasystem andthe development of this metasystem itself, technicaltools are used to analyze and evaluate the objectand significance of the personal contribution of eachdeveloper in special arbitrary units.
• In order to stimulate the development of a marketfor compatible application-based intelligent systemsdeveloped using IMS.ostis Metasystem, for eachsuch intelligent system registered and specified inthe framework of IMS.ostis Metasystem, developersare given remuneration in the used conventionalunits after this application has been tested forsemantic compatibility with other systems devel-oped using the IMS.ostis Metasystem. At the sametime IMS.ostis Metasystem becomes a platform foradvertising and distribution of intelligent systemsdeveloped with its help.
• Stimulating the development of the IMS.ostis Meta-system is as follows. Participation in the develop-ment of the IMS.ostis Metasystem is open, for whichit is sufficient to register accordingly. The copyrightof each developer of IMS.ostis Metasystem is pro-tected and each of his contributions, depending onhis value, is automatically measured and recordedin the conventional units used.
• Participation in the development of IMS.ostis Meta-system can take a variety of forms (in the simplestcase, it can be an indication of specific errors,specific difficulties that the user has encountered,the formulation of specific wishes; a more compli-cated contribution is to add to knowledge base ofnew metasystem knowledge, new components in thelibrary of reusable components). At the same time,the author of a new reusable component includedin the IMS.ostis Metasystem library can choose any
48
license for its distribution and, in particular, assignit any price.
• The use of IMS.ostis Metasystem by registeredusers is free to use with them. In the commercialdevelopment of applied intelligent systems, the costof each access to the IMS.ostis Metasystem is quiteaffordable, but significantly reduced, depending onthe level of user activity in the development ofIMS.ostis Metasystem. This is another mechanismto stimulate participation in the development ofIMS.ostis Metasystem.
Thus, the specified principles of the IMS.ostis Meta-system provide on an ongoing basis the involvementof unlimited scientific, technical and financial resourcesand, in particular, unlimited scientific, technical andfinancial resources to develop the market for semanticallycompatible applied intelligent systems attracting anyprofessionals who want to participate in this open project.
XI. A FAMILY OF VARIOUS OPTIONS FORIMPLEMENTING A UNIVERSAL INTERPRETER OF
SEMANTIC MODELS OF COMPUTER SYSTEMS
universal interpreter of sc-models of computer systems= typical built-in basic ostis-system= built-in empty ostis-system= universal interpreter of sc-models of ostis-systems= universal basic ostis-system, providing simulation of
any ostis-system by interpreting the sc-model of thesimulated ostis-system/*the relationship between the simulated anduniversal ostis-system is to a certain extent similar tothe relationship between the Turing machine and theuniversal Turing machine*/
= SCP language program interpreter/*Semantic Code programming*/
= scp-machine
The implementation of the universal interpreter of sc-models of computer systems may have a large numberof options, both software and hardware implemented.The logical architecture of universal interpreter of sc-models of computer systems ensures the independenceof the designed computer systems from the variety ofoptions for the implementation of the interpreter of theirmodels and includes:• semantic graph associative memory (sc-memory,
sc-storage of sign structures represented in the SC-code);
• interpreter of the SCP language which is a basicprocedural programming language oriented to theprocessing of texts of the SC-code stored in asemantic graph associative memory.
A. Hardware implementation of a universal interpreterof semantic models of computer systems
Semantic associative computer
= Hardware-implemented interpreter of semanticmodels (sc-models) of computer systems
= Semantic associative knowledge-driven computer= A computer with a non-linear structurally
reconstructable (graph-dynamic) associative memory,processing of information in which is reduced not toa change in the state of the memory elements, but toa change in the configuration of the connectionsbetween them
= sc-computer= scp-computer= Computer driven by knowledge presented in the
SC-code= Computer oriented on SC-code texts processing
The basic principles underlying the semantic associa-tive computer:• non-linear memory – each elementary fragment
of text stored in memory may be incident to anunlimited number of other elementary fragments ofthis text;
• reconstructable (reconfigurable) memory – the pro-cessing of the information stored in memory is re-duced not only to changing the state of the elements,but also to reconfiguring the connections betweenthem;
• as an internal method of coding knowledge stored inthe memory of a semantic associative computer, weuse a universal (!) method of nonlinear (graph-like)semantic representation of knowledge, which wecalled the SC-code (semantic, semantic computercode);
• information processing is carried out by a teamof agents working over common memory. Each ofthem responds to the corresponding situation orevent in memory (a computer is controlled by storedknowledge);
• there are software-implemented agents whose be-havior is described by in-memory agent-orientedprograms, which are interpreted by the relevantgroups of agents;
• there are basic agents that cannot be implementedprogrammatically (in particular, they are agents ofagent program interpretation, basic receptor agent-sensors, basic effector agents);
• all agents work over common memory at the sametime. Moreover, if for some agent at some pointin time there are several conditions for its usein different parts of memory, different acts of thespecified agent in different parts of memory canbe executed simultaneously (an agent act is anindivisible, consistent process of agent activity);
• to ensure that agent acts that are executed in par-allel in the shared memory do not "interfere" witheach other, for each act, its current state is fixedand constantly updated in memory. That is, each
49
act informs everyone else about its intentions andwishes that other agents should not interfere with(for example, these are various types of locks of theused elements of semantic memory);
• besides, agents (more precisely, acts performed bythem) must comply with "ethics" trying not to harmthemselves to create the most favorable conditionsfor other agents (acts), for example, not to begreedy, to return faster, not to lock extra memoryelements, as soon as possible to release (unlock)locked memory elements;
• the processor and the memory of the semanticassociative computer are deeply integrated and con-stitute a single processor memory. The processorof the semantic associative computer is uniformly“distributed” in its memory so that the processorelements are simultaneously the elements of thecomputer’s memory. Information processing in thesemantic associative computer is reduced to the re-configuration of communication channels betweenthe processor elements, therefore the memory ofsuch a computer is nothing more than a switchboard(!) of the specified communication channels. Thus,the current state of the configuration of these com-munication channels is the current state of theinformation being processed.
XII. EMBEDDED INTELLIGENT SYSTEM FORCOLLECTIVE DEVELOPMENT OF SEMANTIC
KNOWLEDGE BASES
It is known that the development of a knowledge baseof intelligent systems is a very laborious process, in manyways determining the quality of an intelligent system. Itis also obvious that shortening the development time ofthe knowledge base is possible through the organizationof collective development, but it leads to the number ofproblems, for example:
• How within the team of developers of the sameknowledge base to prevent the syndrome of "swan,crayfish and pike" , or the syndrome of "sevennannies" and how to reduce the overhead costsof coordinating their activities to create a qualityknowledge base.
• How to ensure the possibility of including anyalready formalized knowledge into the knowledgebase of any intelligent system (if they are neededthere) without any “manual" adjustments of thisknowledge and thereby completely eliminate the re-development and adaptation of this knowledge.
The quality of the knowledge base is determined byits following characteristics:
• fullness = integrity = no information holes• consistency = correctness = no errors
• relevance = compliance with the current state ofthe environment and the current state of humanknowledge about the environment
• structuring.The design of intelligent systems consists in building a
semantic model of this intelligent system, which includesthe model of the knowledge being processed, variousmodels for solving various classes of problems, andvarious models for the interaction of intelligent systemswith its external environment. In this case, the knowledgebeing processed can be both problem-solving models inthe knowledge base, and models for solving interfaceproblems, which, respectively, should also be part of theknowledge base of intelligent systems.
A set of tools for designing intelligent systems can bedivided into• tools for knowledge base design;• tools for intelligent system solvers design;• tools for intelligent systems interfaces design.At the same time, it is essential to emphasize that the
design of the problem solver of the intelligent systemconsists in the design of knowledge of a special type –the skills and specifications of the agents who interpretthese skills when solving specific tasks. The designof interfaces of intelligent systems is reduced to thedesign of knowledge, which is a semantic model ofan embedded intelligent system, focused on solving ofinterface problems.
Embedded typical intelligent system of complexsupport for knowledge bases design= Embedded typical intelligent system for complex
automation of design, as well as managing theprocess of collective design and improvingknowledge bases of intelligent systems at all stagesof their life cycle
= Intelligent computer-aided knowledge base designsystem
= Embedded intelligent system, supporting the designand improvement of knowledge bases of intelligentsystems at all stages of their life cycle
= Intelligent computer framework of knowledge basesof intelligent systems developed by OSTIS Technology
= System for the collective knowledge basesdevelopment support based on OSTIS Technology
This embedded intelligent system performs:• monitoring of the activities of each participant in
the process of designing knowledge bases, whichis necessary to protect his copyright, to assess thescope and significance of his contribution to theproject activity, to assess his professional qualifi-cations, to qualitatively assign new design works,taking into account his current qualifications andplanned directions of his qualification enhancement,for the implementation of rollbacks, that is, the
50
cancellation of erroneous decisions made by admin-istrators or managers of the projected knowledgebase;
• version control of the designed knowledge base,the implementation of the necessary rollbacks toprevious versions;
• control of performing discipline;• analysis of the current state and dynamics of the
design process, identification of critical situations;• semantic analysis of the correctness of the results
of the design work of all participants;• assessment of the scope and significance of the
activities of each project participant;• assessment of the current status and dynamics of
the development of the qualification portrait of eachproject participant;
• formation of recommendations for improving theskills of each project participant;
• quality control (consistency, integrity, completeness,clearness) of the current state of the designed andimproved knowledge base.
Each participant in the knowledge base design processcan perform various types of design work:• propose a new fragment in the agreed part of
the knowledge base or some adjustment (deletion,modification) in this part of the knowledge base;
• agree or disagree with the proposed correction oraddition to the agreed part of the knowledge base;
• verify, test, review the correction proposed by some-one or add to the agreed part of the knowledgebase and write comments on the finalization of thisproposal;
• propose the wording of a new project task, forexample, to eliminate the indicated contradiction(errors), to fill in the indicated information hole;
• make constructive criticisms to the wording of thenew project task;
• suggest a performer or a group of performers to per-form a project task that is not yet being performed;
• make constructive criticisms to the proposed per-formers of some free project task.
XIII. SCIENTIFIC KNOWLEDGE PORTALS THATFORMALIZE INTERDISCIPLINARY COMMUNICATION
The objectives of the intelligent portal of scientificknowledge are:• Acceleration of immersion of each person in new
scientific areas with constant preservation of theoverall consistent picture of the World (educationalgoal);
• Fixation in a systematized form of new scientificresults so that all the main connections of newresults with known ones are clearly marked;
• Automation of coordination of work on the reviewof new results;
• Automate the analysis of the current state of theknowledge base.
The creation of intelligent portals of scientific knowl-edge, providing an increase in the pace of integration andthe reconciliation of various points of view, is a way tosubstantially increase the pace of evolution of scientificand technical activity.
Compatible portals of scientific knowledge, imple-mented in the form of ostis-systems, included in Ecosys-tem OSTIS, are the basis of the new principles of orga-nization of scientific activity, in which• the results are not articles, monographs, reports and
other scientific and technical documents, but frag-ments of a global knowledge base, the developers ofwhich are freely formed scientific teams consistingof specialists in relevant scientific disciplines,
• use the portal of scientific knowledge is carried out•• to coordinate the process of reviewing new sci-
entific and technical information from scientiststo the knowledge bases of these portals,
•• the process of coordinating the different pointsof view of scientists (in particular, the intro-duction and semantic correction of concepts, aswell as the introduction and correction of termscorresponding to different entities).
The implementation of a family of semantically com-patible scientific knowledge portals in the form of com-patible ostis-systems, included in Ecosystems OSTIS,involves the development of a hierarchical system of se-mantically consistent formal ontologies corresponding tovarious scientific and technical disciplines, with a clearlydefined inheritance of the described entities propertieswith well-defined interdisciplinary connections that aredescribed by the connections between the correspondingformal ontologies and the subject domains they specify.
Implementing scientific knowledge portals as a fam-ily of semantically compatible ostis-systems also meanstrying to overcome the "babel" diversity of scientificand technical languages, not changing the essence ofscientific and technical knowledge, but reducing thisknowledge to a single universal form of semantic knowl-edge in the memory of scientific knowledge portals, i.e.to a form that is sufficiently clear to both ostis-systems,and any potential users.
An example of a scientific knowledge portal built inthe form of ostis-system is the IMS.ostis Metasystem,which contains all the currently known knowledge andskills that are part of the OSTIS Technology.
XIV. CONCLUSION
The main directions of solving the problem of infor-mation compatibility of computer systems are:• semantic information technology, which is based
on the sense representation of information in thememory of computer systems;
51
• self-organizing ecosystem supporting the evolutionand compatibility of computer systems built on se-mantic information technology during the operationof these systems.
Thus, the current stage of development of traditionaland intelligent information technologies marks the tran-sition from modern information technologies to semanticinformation technologies and to the corresponding self-organizing ecosystem consisting of semantic computersystems. The epicenter of the current stage of develop-ment of information technology is to ensure and self-ensure the information compatibility of computer sys-tems and the consistency of their functioning.
Obviously, the pace of development of semantic infor-mation technologies, as well as the market for appliedsemantic computer systems, depends primarily on thenumber of professionals involved in the developmentof these technologies and in expanding the diversity oftheir applications. The most effective form of achievingthese goals is open projects and, above all, an opendevelopment project of IMS.ostis Metasystem, providingan opportunity for everyone to contribute to the devel-opment of semantic information technologies.
The website of the Belarusian Association ofSpecialists in the Field of Artificial Intelligence(http://baai.org.by [20]) provides information on a num-ber of such open-source projects developed and sup-ported by this association of specialists.
ACKNOWLEDGMENT
This work was supported by the BRFFR-RFFR (NoF18R-220).
REFERENCES
[1] V. Golenkov, N. Guliakina, N. Grakova, I. Davydenko, V. Niku-lenka, A. Eremeev, and V. Tarassov, “From training intelligentsystems to training their development tools,” Otkrytye seman-ticheskie tehnologii proektirovanija intellektual’nyh sistem [Opensemantic technologies for intelligent systems], pp. 81–98, 2018.
[2] A. N. Borisov, “Postroenie intellektual’nyh sistem, osnovannyhna znanijah, s povtornym ispol’zovaniem komponentov[Building of intelligent knowledge-based systems with reusablecomponents],” Otkrytye semanticheskie tehnologii proektirovanijaintellektual’nyh sistem [Open semantic technologies forintelligent systems], pp. 97–102, 2014.
[3] T. A. Gavrilova, D. V. Kudryavcev, and D. I. Muromcev,Inzheneriya znanij: modeli i metody : uchebnik [KnowledgeEngineering: Models and Methods: Textbook]. Spb. : Lan’, 2016.
[4] World Wide Web Consortium [Electronic resource]. Mode ofaccess: http://www.w3.org. — Date of access: 24.11.2016.
[5] M. Wooldridge, An introduction to multiagent systems. Chich-ester : J. Wiley, 2009.
[6] V. Tarasov, Ot mnogoagentnykh sistem k intellektual’nymorganizatsiyam [From multi-agent systems to intelligentorganizations]. M.: Editorial URSS, 2002, (in Russian).
[7] I. Z. Batyrshin, A. O. Nedosekin, A. A. Stetsko, V. B. Tarasov,A. V. Yazenin, and N. G. Yarushkina, Nechetkie gibridnye sistemy.Teoriya i praktika [Fuzzy hybrid systems. Theory and practice].M. : FIZMATLIT [M.: PHYSMATLIT], 2007.
[8] A. Kolesnikov, Gibridnye intellektual’nye sistemy: Teoriya itekhnologiya razrabotki [Hybrid intelligent systems: theory andtechnology of development], A. M. Yashin, Ed. SPb.: Izd-voSPbGTU, 2001.
[9] (2018, dec) Artificial general intelligence conference (agi).[Online]. Available: http://agi-conference.org
[10] A. Palagin, “Problemy transdisciplinarnosti i rol’ informatiki[transdisciplinarity problems and the role of informatics]",”Kibernetika i sistemnyj analiz [Cybernetics and systems analysis],vol. 5, pp. 3–13, 2013.
[11] A. Yankovskaya, “Analiz dannykh i znanii na osnovekonvergentsii neskol’kikh nauk i nauchnykh napravlenii [data andknowledge analysis based on the convergence of several sciencesand scientific fields],” in Mezhdunarodnaya konferentsiya"Intellektualizatsiya obrabotki informatsii" (IOI-8) [InternationalConference "Intellectualization of Information Processing" (IIP-8)]. Kipr, 2010, pp. 196–199.
[12] V. Martynov, Semiologicheskie osnovy informatiki [Semiologicalfoundations of computer science]. Minsk: Nauka i tekhnika[Science and technics], 1974.
[13] I. Melchuk, Opyt teorii lingvisticheskih modelej Smysl-Tekst.Shkola «Yazyki russkoj kul’tury», 1999.
[14] I. A. Melchuk, “Kak nachinalas’ matematicheskaya lingvistika[how did mathematical linguistics begin],” Ocherki istoriiinformatiki Rossii [Essays on the history of computer sciencein Russia], pp. 358–370, 1998.
[15] I. Davydenko, “Semantic models, method and tools of knowl-edge bases coordinated development based on reusable com-ponents,” Otkrytye semanticheskie tehnologii proektirovanija in-tellektual’nyh sistem [Open semantic technologies for intelligentsystems], pp. 99–118, 2018.
[16] (2018, Dec) Ims.ostis metasystem. [Online]. Available:https://ims.ostis.net
[17] A. Bashmakov and I. Bashmakov, Razrabotka komp’yuternykhuchebnikov i obuchayushchikh sistem [Development of computertextbooks and training systems]. M: Informatsionno-izdatel’skiidom Filin" [Information and Publishing House Filin]], 2003.
[18] V. Taranchuk, “Vozmozhnosti i sredstva wolfram mathematicadlya razrabotki intellektual’nykh obuchayushchikh sistem[opportunities and means of wolfram mathematica fordeveloping intelligent tutoring systems],” «Nauchnye vedomostiBelgorodskogo gosudarstvennogo universiteta. Seriya:Ekonomika. Informatika» [«Scientific statements of BelgorodState University. Series: Economy. Computer science»], vol. 33,no. 1 (198), pp. 102–110, 2015.
[19] D. Shunkevich, “Agent-oriented models, method and tools ofcompatible problem solvers development for intelligent sys-tems,” Otkrytye semanticheskie tehnologii proektirovanija in-tellektual’nyh sistem [Open semantic technologies for intelligentsystems], pp. 119–132, 2018.
[20] (2018, Dec) Belarusian public association of experts in the fieldof artificial intelligence. [Online]. Available: https://baai.org.by
МЕТОДЫ И СРЕДСТВА ОБЕСПЕЧЕНИЯСОВМЕСТИМОСТИ КОМПЬЮТЕРНЫХ
СИСТЕМ
Голенков В.В., Гулякина Н.А., Давыденко И.Т.,Еремеев А.П.
В работе рассмотрены основные актуальные про-блемы в области разработки современных компьютер-ных систем, в частности – проблема обеспечения ин-формационной совместимости компьютерных систем.Предложен подход к них решению, основанный наиспользовании Открытой семантической технологиипроектирования интеллектуальных систем (OSTIS).
Received 11.12.18
52
Principles of organization and automation ofthe semantic computer systems development
Vladimir Golenkov, Daniil Shunkevich, Irina Davydenko, Natalia GrakovaBelarussian State University Informatics and Radioelectronics
Minsk, [email protected], [email protected], [email protected], [email protected]
Abstract—The work is devoted to the principles of devel-opment of semantic computer systems of a new generationbased on the Open Semantic Technology for IntelligentSystems Design (OSTIS Technology). The advantages of thetransition from traditional computer systems to semanticcomputer systems from the point of view of the designprocess are substantiated, and the advantages of implemen-tation of automation tools for design activities as semanticcomputer systems are considered.
Keywords—design automation tools, semantic technology,semantic network, knowledge base, problem solver, ontol-ogy, multi-agent system
I. INTRODUCTION
At the present stage of development of computertechnologies, various types of design automation systems(CAD systems) are widely used in almost all productionareas. The use of systems of this kind is relevant both inthe production of material objects and in the developmentof computer systems. Modern CADs allow automatingmany processes related to both directly designing anobject and its development and implementation, and, as aresult, can significantly reduce production time and costof the product, as well as minimize the number of errorsassociated with human factors. An important directionof the development of CADs is their intellectualization,which imposes fundamentally new requirements on thedevelopment of CAD technologies.
In a number of previously published works, the authorsproposed the concept of the Open Semantic Technologyfor Intelligent Systems Design (OSTIS Technology) [1],[2], focused on the development of computer systemsof the new generation (first of all – hybrid intelligentsystems [3]) which will be called semantic computer sys-tems or ostis-systems, if it is necessary to emphasize theircompliance with the standards of OSTIS Technology.The model of hybrid knowledge bases of ostis-systemsand models for representing various types of knowledgewithin the framework of such a knowledge base [4], aswell as model of a hybrid problem solver, which allowsto integrate various problem solving models [5], werealso proposed.
The main requirement for OSTIS Technology is toensure the possibility of joint use within the ostis-systemsof various types of knowledge and various problems
solving models with the possibility of unlimited expan-sion of the list of knowledge used in ostis-system andproblem solving models without significant labor costs.The consequence of this requirement is the need toimplement the component approach at all levels, fromsimple components of knowledge bases and problemsolvers to whole ostis-systems.
To meet these requirements, the most important taskis not only the development of appropriate ostis-systemsmodels and their components, but also the developmentof an integrated methodology and appropriate tools forautomating the construction and modification of ostis-systems.
Thus, within the framework of this work, attention ispaid to the principles of organizing and automating thedevelopment process ostis-systems, which underlie therelevant methodology and tools, including the principlesof regulation and stimulation of activities aimed at de-veloping ostis-systems and their components, and alsoconsidered the main advantages of the transition fromtraditional computer systems to ostis-systems from thepoint of view of the design process and maintenance ofsuch systems.
However, the transition from traditional computer sys-tems to ostis-systems will allow not only to obtain anumber of advantages associated with such key proper-ties of ostis-systems, such as hybridity, modifiability andlearnability, which are discussed in detail in the aboveworks, but it will also allow to bring to a fundamentallynew level the processes of designing and maintainingof computer systems and the degree of automation ofsuch processes. In this case, it is assumed that the build-ing automation and modification tools for ostis-systemsshould be implemented as a ostis-system themselves andintegrated with the system being developed, which willgive a number of additional benefits.
Thus, within this work attention is paid to the prin-ciples of organizing and automating the developmentprocess of ostis-systems, which underlie the relevantmethodology and tools, including the principles of regu-lation and stimulation of activities aimed at ostis-systemsand their components developing, and also consideredthe main advantages of the transition from traditional
53
computer systems to ostis-systems from the point of viewof the design process and maintenance of such systems.
II. INTELLECTUALIZATION OF DESIGN AUTOMATIONTOOLS
Much attention in modern literature is given to var-ious approaches to the construction of intelligent CADsystems. Unfortunately, in many cases, intellectualizationrefers to adding the simplest adaptive functions («in-telligent cursor», «intelligent menu», etc.), however, ananalysis of the sources revealed key areas in the field ofthe intellectualization of CADs, within which there aresignificant results.
One of the most important and most promising areas inthe field of building intelligent CAD is generative design[6], which assumes that the computer system itself actsas an active participant in the design process. Accordingto the concept of generative design, the designer sets therequired minimal description of the parameters of thedesigned object, after which the system independentlygenerates the initial version of the designed object model,which is further refined and updated in dialogue with thehuman designer. Important for the development of thisdirection was the introduction at first of CNC machines,and then 3D printing technologies into mass production,making it possible to manufacture objects of a muchmore complex shape based on their detailed model.Taken together with the development of computing pow-ers this made it possible to solve various optimizationproblems using CAD, in particular, the problem of topo-logical optimization (eliminating unnecessary materialfrom a part while maintaining key properties such asmaximum load).
Autodesk Dreamcatcher [7] is the most advanced andcurrently popular tool implementing the concept of gen-erative design, which allows to develop designs of partsfor various products based on the design intent expressedby people in natural language and then modify projectsto simplify their production on a specific equipment.
Another important area of CADs intellectualizatio,which is significantly less developed in comparison tothe one discussed earlier, is the direction suggestingthat CAD should also perform the learning system [8]functions. It is important to note that both the learning ofthe designer and the learning of the system itself in theprocess of work are considered. In turn, the designer’slearning can also be considered in two aspects: learningin working with CAD systems, that is, studying thefunctionality and principles of working with a particularsystem, as well as learning in the actual subject domainin which the design is carried out, that is, studying thedetailed aspects of the designed objects, their purpose,design features, etc. It is obvious that the developmentof this direction imposes additional requirements on thetechnologies underlying such systems, in particular, re-
quires the coordinated use of heterogeneous informationand various models of information processing.
Another important area of intellectualization is sim-ulation modeling. Simulation can be widely used atdifferent stages, in particular:• modeling the behavior of the object of development
under the influence of various factors, in a varietyof external environment, etc., which is especiallyimportant when developing complex expensive sys-tems designed to work in unpredictable conditions;
• project management modeling, which can be usedfor training or testing personnel, assessing potentialrisks when choosing a specific management strat-egy, working out certain practices and skills underconditions similar to a real project [9];
Obviously, to realize the possibility of comprehensivesimulation, it is necessary to have tools that allow, on theone hand, to describe in details complex heterogeneousobjects from different points of view, i.e. in fact, integratewithin the framework of a unified system various typesof knowledge, as well as various approaches to theinterpretation of such descriptions. In addition, it isnecessary to have the ability to easily modify the existingmodels, in particular, it should be easy to change thenumber and types of influencing factors, it should beeasy to change the principles of behavior of objects ofthe environment and the simulated objects themselves,etc.
It is important to note that simulation modeling canbe the basis for the use of other models and methods ofartificial intelligence. For example, the possibility of alarge number of simulation launches and the accumula-tion of certain information about these simulations canbe the basis for their further analysis and application ofmachine learning methods.
The next stage in the development of productionsystems in general is the transition from CADs tomore general PDM-systems (Product Data Management),and further to complex PLM-systems (Product Lifecy-cle Management) [10], as well as CALS-systems andtechnologies (Continuous Acquisition and Lifecycle Sup-port).
The construction of such integrated information sys-tems requires the unification (standardization) of hetero-geneous information. To solve this problem, the onto-logical approach is currently widely used both in thedevelopment of software systems [11], [12], and in otherareas [13], [14].
Thus, various kinds of intellectualization of designautomation tools require solving the compatibility prob-lem of various information representation and processingmodels, a specific list of which for different systemsmay differ significantly, since it depends on the designobject, requirements for functionality of the tools, etc. Inaddition, the development of such funds, including those
54
associated with changes in the design object, requires areduction in the laboriousness of modifying these tools,which is also currently a serious problem.
In this paper, it is proposed to solve these problems byusing unified models for the representation and process-ing of information proposed in the OSTIS Technology, inparticular, models of hybrid knowledge bases and hybridproblem solvers. Next, we consider in more detail theprinciples of building automation tools that implementthese principles.
III. ARCHITECTURE AND FEATURES OF THEDEVELOPMENT OF SEMANTIC COMPUTER SYSTEMS
A. Architecture of semantic computer systems
Lets consider the features of the ostis-systems archi-tecture, affecting the design process and the principlesof appropriate automation tools constructing.
As a basis for knowledge representation in the frame-work of OSTIS Technology, a unified version of codinginformation of any kind based on semantic networks isused, named SC-code [1]. Elements of SC-code text (sc-texts) are named sc-elements, among which, in turn, aresc-nodes, sc-arcs and sc-edges. As part of the technology,several universal variants of visualization of SC-code,such as SCg-code (graphic variant), SCn-code (nonlinearhypertext variant), SCs-code (linear string variant).
Each ostis-system consists of a complete model of thissystem, described by means of SC-code (sc-model ofcomputer system) and sc-model interpretation platform,which in general can be implemented both in softwareand in hardware [2]. This ensures complete platformindependence of ostis-systems.
In turn, the sc-model of computer system is con-ventionally divided into sc-model of knowledge base,sc-model of problem solver and sc-model of computersystem interface (including user interface, interface withthe external environment and interface with other ostis-systems), as well as the model of abstract semanticmemory (sc-memory), in which the SC-code constructsare stored, and, accordingly, all the listed sc-models(figure 1).
The principles of building of sc-models of knowledgebases and sc-models of problem solvers are discussed inmore detail in the papers [4] and [5], respectively.
The sc-model of knowledge base is based on suchbasic principles as the distinguish of the hierarchicalsystem of subject domains and ontologies (including thepresentation level meta-ontology and the family of top-level ontologies that are part of each developed ostis-system), as well as the distinguish of structures (signsof entire fragments of the knowledge base), which canbe subsequently described in the same knowledge base.The use of these and other principles provides suchimportant properties as the ability to represent knowledgeof various types in the knowledge base, ease of making
sc-models interpretation platform
ostis-system
sc-memory
sc-model ofknowledge base
sc-model of problemsolver sc-model of interface
sc-model of ostis-system
Figure 1. Ostis-system architecture
changes to the knowledge base, including the possibilityof expanding the set of knowledge types used, as well asthe possibility of structuring the knowledge base accord-ing to an arbitrary set of features and the possibility ofrepresentation in the knowledge base of meta-knowledgeof an arbitrary level.
The sc-model of problem solver is based on theprinciple that the solver is treated as a hierarchical systemof agents that react to situations and events in sc-memory(sc-agents) and interact with each other exclusively byspecification of the information processes performed bythe corresponding agents in the sc-memory. Such sc-agents can be atomic, i.e. those for which the programof their actions is specified and non-atomic, i.e. thosethat are decomposed into simpler sc-agents. Classesof functionally equivalent sc-agents are called abstractsc-agents. Each abstract sc-agent has a correspondingspecification that contains, at a minimum, the initiatingcondition of sc-agent and the sc-agent implementationdescription depending on whether it is atomic or non-atomic sc-agent.
Further, speaking about the knowledge base, problemsolver and user interface, we assume that we are talkingabout the sc-model of knowledge base, sc-model of prob-lem solver and sc-model of user interface, respectively.
The sc-model separation into components is ratherconditional, since an important architectural feature ofostis-systems is the fact that both the solver and thesystem interface are in fact part of its knowledge base.This is achieved through the following principles:• all agents that are part of the solver (including the
programs of agents), all the information processesthey perform are described in the knowledge baseby means of SC-code. This possibility, in turn, isachieved due to the presence within the frameworkof OSTIS Technology of programming languageSCP, the programs of which are written using SC-code;
• the ostis-system interface is treated as a subsystembuilt according to the same principles, that is,
55
having its own sc-model of knowledge base and sc-model of problem solver, which in turn are basedon the corresponding principles discussed above.
Thus, the most important feature of the developmentof ostis-systems is that development of ostis-systemactually comes down to development of its knowledgebase. When developing the components of the problemsolver and the interface, their features are taken intoaccount, however, the general mechanism for making anychanges to the ostis-system becomes unified.
B. General typology of project actions of semantic com-puter systems developers
In the general case, when developing ostis-systems (aswell as many other computer systems), the followingtypes of project activities can be performed:• synthesis (generation) of components and systems
with specified properties•• search for the closest components in compo-
nents library;•• adjustment of the specified (for example, found)
component in order to obtain the specified prop-erty
•• assembly of large parts;• integration of the developed component into the
system for which the component is intended;• analysis of the developed component or system•• analysis of correctness (absence of errors and
contradictions);•• analysis for compliance with the required char-
acteristics, including testing and test generation;•• integrity analysis (completeness);•• clearness analysis (absent of excesses);•• value analysis;•• evaluation of the project workload;
• specification (description, documentation) of thedeveloped component or system;
• control of project discipline (adherence to workschedule);
• design management (assignment of performers anddeadlines for specific project tasks);
• developer coordination;• version control;• analysis of the contribution of each developer to the
overall result;• stimulation of project activities.
IV. EXISTING APPROACHES TO THE ORGANIZATIONOF THE COMPUTER SYSTEMS DEVELOPMENT
Despite the architectural features, each ostis-systemis a computer system, and therefore, when designingostis-systems, it is necessary to take into account currenttrends in the development of computer systems and therequirements for the corresponding automation tools.
In this section, we consider existing approaches tothe organization of the development process of computersystems depending on the type of development object:
• general methodologies for developing computersystems, the object of development in general is anycomputer system, including the semantic computersystem;
• means of automating the computer systems de-velopment, the object of development is a com-puter system built on the basis of traditional com-puter technologies. Many approaches implementedin such tools can also be used in the developmentof semantic computer systems; however, in a ready-made form, these tools are not focused on thedevelopment of such systems and do not take intoaccount their features;
• tools for developing knowledge bases and ontolo-gies, the object of development is the componentsof knowledge bases, first of all, ontologies;
• means of component development of intelligentsystems, the object of development is intelligentcomputer systems.
A. Modern general methodologies for computer systemsdevelopment
With the development of information technologies inthe last decade, the Agile [15], [16] family of softwaredevelopment methodologies has gained the most popu-larity. Most Agile methodologies belong to the so-calledlightweight methodologies and are contrasted with theclassic heavyweight, such as, for example, the cascade(waterfall) model. It can be said that modern Agilemethodologies actually integrate the best of the ideasunderlying the more traditional methodologies (spiral,iterative, cascade, etc.), taking into account the pecu-liarities of the current stage of software systems de-velopment, first of all - the need for substantial morerapid adaptation ever-changing requirements, as well asthe need for relevant workable versions of the systembeing developed. These features are fully valid for ostis-systems, however, due to their architectural features,many principles of Agile methodologies for such systemscan be implemented easier than for traditional computersystems, which will be discussed in more detail below.
B. Existing automation tools for the development ofcomputer systems
Most modern computer-aided design tools do not limitusers to using any particular methodology (although thereare tools that implement a specific methodology, forexample, Trello).
In this section, we briefly review the main classes oftools aimed at solving some particular problems in thefield of computer systems development.
56
1) Version control systems: Version control systemsare currently used in the development of almost anysoftware systems and give developers the following mainfeatures:• save project versions with the ability to rollback to
any previous version if necessary;• fixing the authorship of each change, the date and
time of the change, as well as the ability to specifythe sense and reason for the change;
• access control and the possibility of making changesto the main project for different developers;
• many version control systems allow to create dif-ferent versions of the same document, if necessary.
An overview of modern version control systems andtheir comparative analysis is given, for example, in [17].
2) Issue tracking systems, bug-tracking systems: Sys-tems of this class allow developers to fix current projecttasks (including errors correcting and other problems),assign performers and deadlines for tasks, indicate thestatus and priority of tasks. Modern systems of this classprovide wide opportunities for discussing tasks, assessingthe contribution and activity of developers, etc.
A comparative analysis of systems of this class is givenin [18].
3) Project management systems: Project managementsystems are in many ways similar to bug tracking sys-tems, but unlike them, as a rule, they are not focusedon developing only software products. In addition, thekey difference in developed project management systemsis the availability of tools for estimating project devel-opment deadlines, development of plan implementationmonitoring tools, visualization of project activities in theform of generally accepted diagrams, etc., that is, theemphasis in such systems is transferred to managementand control design process.
A list of popular systems of this class with a briefdescription of their capabilities is given in [19].
4) Verification automation systems: Verification au-tomation systems can be divided into the followingclasses:• continuous integration systems that integrate with
the version control system, build a project for eachnew version or according to a schedule, automati-cally perform a number of embedded tests, and iferrors occur, immediately inform the developers;
• test automation systems that allow to automatepart of the manual work in the process of testing ofthe developed product and its components.
Lists of systems of this class with specifications aregiven, for example, in [20], [21].
5) Project hosting systems: Project hosting systemsgive developers the opportunity to place repositories withthe code of their projects in the cloud, and administerthem. Each system of this kind is a complex system ca-pable of working with at least one version control system,
and also, as a rule, integrates the auxiliary systems of allthe classes listed above (project management, trackingerrors and tasks, automatic verification). Many systemsof this kind also provide ample opportunity for projectdocumentation.
A comparative review of the most popular systems ofthis class today is given, for example, in [22].
Thus, the systems of these classes solve problems thatare relevant in the development of computer systems ofany kind, including ostis-systems. Due to this the task ofdeveloping tools that would solve these problems whendesigning ostis-systems taking into account the specificsof such systems becomes urgent. The implementationof such tools on the basis of OSTIS Technology willsignificantly simplify the integration of subsystems thatsolve different tasks, which, in turn, will expand thefunctionality of such tools, which will be discussed inmore detail below.
C. Knowledge base and ontology development method-ologies
As mentioned earlier, the development of the ostis-system comes down to the development of its knowledgebase, and therefore it is advisable to consider existingapproaches to the development of knowledge bases, aswell as appropriate tools. At the same time, in modernliterature, when analyzing methods for knowledge basesdevelopment, the focus is on analyzing methodologiesfor developing ontologies, which are the basis of mod-ern knowledge bases, including knowledge bases ostis-systems.
There are many papers devoted to the review of variousapproaches and methodologies for ontology design [23],[24].
In [25], a variant has been proposed for the classifi-cation of existing methodologies for the development ofontologies, based on the most essential features, whichinclude:• team development support;• degree of dependence on the toolkit;• type of ontology life cycle model used;• possibility of formalization;• ability to reuse knowledge base components;• strategy for distinguishing of subject domain con-
cepts;• ability to support compatibility of developed ontolo-
gies.Among the main methodologies that have been most
developed to current date and have become basic inthe field of creating ontologies of subject domains,the following can be singled out: Ushold and King’sskeletal methodology [26], Gruninger and Fox method-ology (TOVE) [27], METHONTOLOGY [28], [29],On-To-Knowledge (OTK) [30], KACTUS [31], DILI-GENT [32], SENSUS [33] and UPON [34]. Their com-
57
parative characteristics in accordance with the aboveclassification are given in [25].
Analysis of the reviewed methodologies shows thatnone of them is complete enough, and all proposedsolutions are not unified. Most methodologies do notsupport the joint development of knowledge bases, thecompatibility support for the knowledge bases beingdeveloped and, as a result, the support for the reuse of al-ready developed knowledge bases and their components.
In addition, the overwhelming majority of knowledgebase development methodologies describe the develop-ment process in general terms, not regulating the actionsof participants at each stage of ontology development, notspecifying the principles of matching new concepts withexisting ones, the subjective influence of developers ishigh. Thus, the problem of compatibility of componentsof knowledge bases remains relevant when using eventhe most developed methodologies.
D. Knowledge base and ontology development tools andknowledge base development projects
Let us consider in more detail some of the mostcommon knowledge base development tools availabletoday.
Wiki-technologyWiki-technology allows to accumulate knowledge that
is presented in an interoperable form, providing navi-gation through knowledge. It is possible to use Wiki-Technology for projects of any scale and thematic fo-cus (from open electronic encyclopedias, to referencesystems of various enterprises and educational institu-tions) [35], [36].
Wiki-Technology provides its users tools for storingand structuring text, hypertext, files and multimedia.Wiki-Technology uses the MediaWiki [37] platform asa tool, which allows to perform information interaction,providing access to information resources to all partic-ipants in the system development process, organizingmanagement and monitoring of the development [38].The advantages of this technology include the simplicityof Wiki markup, communication capabilities that arerealized through joint editing of pages, as well as throughelectronic discussions in the Wiki or additional mediasuch as chat or forum, the design nature of the work,cooperation, the formation of a single product of joint ac-tivities meaningful interaction, knowledge sharing, eval-uation and continuous improvement of work [35].
The influence of the Semantic Web on such projects isconstantly increasing, as a result, Wiki-sites engines havebeen created that support the ontological representationof knowledge and semantic markup of resources usingSemantic MediaWiki [39]. These tools allow you toinclude semantic annotations in Wiki markup in the formof OWL and RDF and explicitly separate structured andunstructured information [35].
In addition to these advantages, the Wiki as a technol-ogy has several disadvantages: duplication of informationon different pages, the impossibility of structuring knowl-edge due to the lack of a hierarchy of hyperlinks and thelack of unification of the presentation of information,the lack of automatic verification. In addition, Wiki-Technology is currently designed to work only withstructured natural-language texts, thus, based on thistechnology, it is not possible to build knowledge basesof intelligent systems, since informal text is unsuitablefor automatic processing to the extent necessary forsolving various problems, for example, logical inferenceproblems.
However, many ideas of Wiki-technology can beadapted for the collective development of knowledgebases.
Software environments for ontology constructionExisting software for building knowledge bases (on-
tologies) are conditionally divided into three [40] groups:1) Ontology creation tools. This class of tools supports
the process of creating a knowledge base ”from scratch”.In addition to editing and browsing, tools provide supportfor ontology documentation, ontology import/export intovarious formats and languages, and ontology librarymanagement.
These include: Protégé [41], NeON [29], Co4 [42],Ontolingua [43], OntoEdit [30], OilEd [44], We-bOnto [45] etc. A brief description of these and othertools can be found in [46], [47], [48].
2) Tools for displaying, aligning and combining ofontologies. This class of tools helps users find the sim-ilarities and differences between the original ontologiesand create a resultant ontology that contains elements ofthe original ontologies. The author [47] divides them intosubgroups according to the following features:• to combine two ontologies in order to create the
new one (PROMPT [49], Chimaera [50], On-toMerge [51]);
• to determine the conversion function from one on-tology to another (OntoMorph [52]);
• to define the mapping between concepts in twoontologies, finding pairs of corresponding concepts(for example, OBSERVER [53], FCA-Merge [54]);
3) Ontology-based annotation tools. The most impor-tant condition for implementing the goals of the SemanticWeb is the ability to annotate Web resources with meta-information. For this reason, recently ontology engineer-ing tools include ontology-based annotation tools. Theseinclude: MnM [55], SHOE Knowledge Annotator [56],etc.
In the context of solving the tasks set in the frameworkof this work, it makes sense to consider in detail onlythe first class of tools.
In ontological engineering, any ontology is consideredas the result of coordinated activities of a group of
58
specialists on a model of a certain field of knowledge.Based on this, with the development of methods andtools in the field of knowledge engineering, an increasingattention has been paid to the instrumental support of theprocess of collective development of ontologies, withinwhich there are several basic tasks [57], [58]:• management of interaction and communication be-
tween developers;• access control to current results of joint design;• copyright fixation for expert knowledge passed to
the public;• design error detection and error correction manage-
ment;• competitive change management.Currently, to solve these problems, there are already
several fairly well-developed approaches and appropriatetools. Among them are the following:• Collaborative Protege [59];• NeOn project [29];• infrastructure for joint development of consistent
knowledge bases Co4 [42].The main disadvantages of the considered tools in-
clude:• the lack of developed tools for automatic editing
and verification of knowledge bases, including theassessment of completeness and redundancy;
• lack of a single mechanism for the collective cre-ation of knowledge bases, including means of co-ordinating changes between developers of differentlevels of responsibility, a typology of developerroles;
• insufficient level of extensibility of developmenttools.
E. Analysis of the means of component development ofintelligent systems
The use of ready-made components is the most im-portant way to reduce the time and complexity of thedevelopment of computer systems, and reduce the pro-fessional requirements for their developers.
The issues of component design of intelligent systemsand, in particular, knowledge bases and problem solversare discussed in the works [60], [61], [62], [63]. Whencreating the first systems based on knowledge, it wasassumed that these systems would ideally solve theproblem of reusable components, however, developersfaced a number of problems that are relevant to date [64].
Many researchers and developers determine the avail-ability of ontological libraries as an important com-ponent of the Semantic Web [65] infrastructure. Thefirst libraries and collections of ontologies were devel-oped within such projects as Ontolingua server [43],DAML [44], Protégé ontology library [41], Ontariaontology directory [66] and SchemaWeb [67]. Manyof these projects are not currently supported, but they
are being replaced by a new generation of ontologylibraries [68].
However, as shown in [69], [70], the majority ofontologies developed based on the Semantic Web stan-dards are not consistent with each other and, therefore,cannot be reused as components of knowledge bases, aswas supposed by the developers of the Semantic Webstandards.
The problems in the component design of knowledgebases include the following:
• many components use the developer’s language(usually English) to identify concepts, and it isassumed that all users will use the same language.However, for many applications, this is unaccept-able – developer-only identifiers should be hiddenfrom end users, who should be able to choose alanguage for identifiers that they see [64];
• lack of unification in the principles of representingdifferent types of knowledge within one knowl-edge base, and, as a result, the lack of unifica-tion in the principles of identifying and specifyingreusable components leads to incompatibility ofcomponents developed in the framework of differentprojects [60];
• lack of search engine components that meet thespecified criteria.
To date, a large number of knowledge bases havebeen developed in the different subject domains [71].However, in most cases, each knowledge base is devel-oped separately and independently from the others, in theabsence of a unified formal basis for the presentation ofknowledge, as well as unified principles for the formationof systems of concepts for the described subject domain.In this connection, the developed bases are, as a rule,incompatible with each other and not suitable for reuse.
In turn, in the development of problem solvers thereare a large number of specific implementations, but thecompatibility issues of different solvers for solving oneproblem are practically not considered in the literature.
There are a number of works that solve problems of ac-cumulation and reuse of components of problem solvers[60], as well as the development of a unified platformfor integrating various models of problem solving [72],but the problem of their compatibility is still relevant.
Thus, the problem of the development of commonunified principles for the distinguishing and specificationof reusable components of intelligent systems and theformation of a library of such compatible componentsis still relevant. In this case, reusable components ofdifferent levels of complexity can be distinguished - fromspecifications of single concepts and single knowledgeprocessing agents to reusable ontologies, ontology sys-tems, and problem solvers.
59
V. REQUIREMENTS FOR THE METHODOLOGY ANDDEVELOPMENT TOOLS FOR SEMANTIC COMPUTER
SYSTEMS
This section considers the requirements formulated onthe basis of the analysis of modern methods and toolsfor developing computer systems, as well as modernapproaches to the development of intelligent systems, andin particular, knowledge bases.
Given that the development of ostis-system comesdown to the development of its knowledge base, therequirements for ostis-systems development automationtools include:• refusal to edit the source code of the knowledge
base (in any external languages) in favor of directlyediting the knowledge base stored in the sc-memoryby means of the appropriate editors. The mainadvantages of this approach are as follows:•• the ability to automate the verification and edit-
ing of the knowledge base stored in memory;•• the possibility of automating the process of inte-
grating new knowledge (first of all, identifyingand eliminating synonymous fragments) into theknowledge base being developed;
•• the ability to edit the knowledge base (as aresult, the problem solver) directly during theoperation of the system;
•• possibility during the collective development toallocate, if necessary, fragments of the knowl-edge base of arbitrary configuration and appealto them in the process of discussion and agree-ment;
• there should not be any fundamental restrictions onthe top level development methodology used, thenomenclature of the distinguished roles of devel-opers, models of organization and management ofthe development process. At the same time, dueto its openness, OSTIS Technology is focused onthe development of primarily open-source projects,which also have a number of features from the pointof view of the designing organization [16];
• possibility of joint development of a knowledgebase by the development team (including distributedteams), including the possibility of discussing(agreeing on) and administering the changes, ifnecessary, the ability of third-party subject domainexperts help in need to solve some contradictions.In addition, with an increase of the knowledge basesize, it becomes important to organize the hierar-chical administration of the knowledge base, withindicating the responsibilities of each administrator;
• the possibility, when experts have opposing pointsof view, in the process of agreeing on any fragmentsof the knowledge base to fixate and/or resolve thesecontradictions. This problem is particularly relevantin scientific projects where the truth of certain
judgments can be confirmed or refuted for a longtime;
• independence of the methodology and knowledgebase development tools (except for external lan-guage editors) on which external language is usedto develop the current knowledge base fragment(SCn, SCg, etc.), as well as on which identificationlanguage of sc-elements (English, Russian, etc.) iscurrently in use;
• availability of convenient and accessible tools ofmanual editing of the knowledge base using externallanguages;
• fixations of the entire history of changes in theknowledge base with the obligatory indication ofauthorship and the time of making each change,including both changes to its subject part (intendedfor the end user), and any changes associated withthe development process, including commenting,approval or rejection of the proposed knowledgebase changes, etc.;
• ensuring the integrity (consistency) of the knowl-edge base being developed at each point in timeduring its development, while in the early stagesof development of automation tools, the degree ofhuman participation in integrity ensuring can besignificant and subsequently decrease. This require-ment is largely due to the pursuit of the ideas ofAgile-methodologies;
• reflexivity of the ostis-system development process,suggesting that the developed ostis-system itselfbecomes an active participant in the developmentprocess, that is, it can analyze current project tasks,processes aimed at its development and any relatedinformation.
In addition, when developing ostis-systems, the fol-lowing requirements stay valid, which are valid whendeveloping computer systems of any kind:
• availability of means for assessing the contributionof each participant to the development process,which allows calculating the amount of materialremuneration of the development process partici-pants, taking into account the overall activity ofvarious participants when determining their rolesand privileges within the hierarchy of developers,etc.;
• availability of tools and mechanisms to implementthe process of material incentives for developers, inparticular, investing in certain areas of development,micro-investment;
• availability of design management tools, including,at a minimum, means of current tasks fixation, theirpriorities, dependencies, deadlines and performers,tools of controlling the process of performing tasks,etc.;
• focus on the use of reusable components of dif-60
ferent degrees of complexity, involving both theorganization of the accumulation process, the spec-ification and the search for components within therelevant library, and the organization of the processof integrating components from the library into thesystem being developed, as well as including newcomponents into the library;
VI. THE PROPOSED APPROACH TO THEORGANIZATION OF SEMANTIC COMPUTER SYSTEMS
DEVELOPMENT
Taking into account the formulated requirements, thefollowing basic principles were used as the basis for theproposed approach to the development of ostis-systemsand the means of automation of this process:• tools for the ostis-systems development process
automation are also implemented as ostis-systemsand are built according to the same principles. Thisprinciple allows:•• to ensure the modifiability of the tools them-
selves, as well as the hybridity of such tools,which is expressed in the possibility of com-bining within such tools different approaches(methodologies) to the development and verifi-cation of the knowledge base;
•• provide information support to developers andtheir learning directly in the process of workingwith automation tools due to the possibility ofpresenting in the knowledge base of such toolsthe information about the project, developmentmethods, architecture and principles of oper-ation of the tools themselves in a structuredmanner, as well as the ability to ask differentquestions;
•• the possibility of implementing tools for analyz-ing information about a project with the possi-bility of permanent expanding the functionalityof such tools if necessary;
•• the possibility of implementing tools of vari-ous kinds of modeling with the possibility ofpermanent expanding the functionality of suchtools if necessary;
• development process automation tools ostis-systemsare embedded as a subsystem in the developedostis-system, thus the knowledge bases and problemsolvers of the developed system and automationtools are integrated. This ensures the reflexivity ofthe development process, that is, the possibility ofthe participation of the developed system in thedevelopment process itself. Due to this principle thefollowing advantages appear:•• eliminating the need to use the source code of
the knowledge base and eliminating the stageof assembling and deployment of the system, aswell as the need to restart the system to make
changes. Thus, at any time there is a working,serviceable version of the ostis-system, whichfully complies with the principles of Agile;
•• eliminating the need to document the systembeing developed (which is also consistent withthe principles of Agile), since the knowledgebase itself contains all the necessary informa-tion, which is accessed by the same tools aswhen solving any other problem;
•• when forming project tasks and discussingthem, it becomes possible to appeal directlyto fragments of the system’s knowledge base,which have an arbitrary configuration, whichmakes it more flexible, for example, in theprocess of specifying problem parts of theknowledge base;
•• communication between developers is carriedout on the same principle as the solution ofproblems by a team of agents, that is, all actionsperformed (including actions for the formationof natural language messages) are recorded ina common memory. At the same time, theauthorship of the action, the object of the action,if any, and any other necessary information isindicated. This approach does not fully corre-spond to the principle of personal communica-tion adopted in Agile, however, on the one hand,it does not exclude the possibility of personalcommunication of developers, and on the otherhand, it makes it possible to develop a projectby a distributed team of developers, includingthe case of open-source development;
•• it is possible not only to automatically detecterrors and problems, but also to automaticallygenerate tasks for correcting them within theframework of automation tools;
•• it becomes possible to record in the knowledgebase not only the authorship of certain frag-ments, but also the whole process of agree-ing and discussing the changes made, and, ifnecessary, even to have contradictory fragmentsin the knowledge base, the truth of each ofwhich cannot be installed exactly. At the sametime, the opinions of various experts who tookpart in the discussion, the arguments for andagainst, etc. are recorded. At the same time,solving problems in a knowledge base contain-ing conflicting information remains possible byspecifying the solution context for each prob-lem, that is, that area of the knowledge basethat is considered correct when solving a spe-cific problem. Thus, the proposed approach notonly makes the discussion of the changes openand transparent, but also makes it possible torecord in the knowledge base simultaneously
61
any points of view on the same fragments,including contradictory.
The implementation of these principles implies theimplementation of subsystems of the ostis-systems de-velopment automation tools, similar to all classes ofexisting automation tools for the development processof traditional computer systems. However, this task israther time-consuming, and it would be inappropriateto make the possibility of ostis-systems developmentdependent on it. At the present stage of development,OSTIS Technology for solving many particular problemsrelated to ostis-systems design, traditional tools can beused (some of mentioned advantages will be absent atthis stage), while with the development of technologythere will be a step-by-step transition from traditionaltools to automation tools built on the basis of OSTISTechnology.
Besides the development automation tools ostis-systems embedded in the system being developed,IMS.ostis Metasystem [73] plays an important role inthe development process.
IMS.ostis Metasystem is also an ostis-system andsolves the following main tasks:
• informational support for ostis-systems developers,which assumes that the knowledge base of themetasystem in each current time is a completeformal description of the current version of OSTISTechnology (including a complete description ofthe metasystem itself, all the main models usedin the technology, and also methods and tools fordeveloping components of ostis-systems), as wellas the availability of navigation tools in such aknowledge base;
• the accumulation of ostis-systems development ex-perience and the implementation of the componentapproach, which is expressed by the presence oflibraries of reusable components ostis-systems, aswell as the means of component specification andsearch tools components based on their specifica-tions.
To implement the principles discussed earlier, it isproposed to use an integrated ontological approach tothe design of computer systems, discussed in [74]. Thisapproach involves the construction of two ontologicalmodels:
• ontological model of the design object, in this case,ostis-system;
• ontological model of project activities aimed at thedevelopment of appropriate design objects;
Ontological models of ostis-systems and their com-ponents were considered in detail in the previouslymentioned works [74], [4], [5].
Building an ontological model of a project activityaimed at ostis-systems development involves the devel-
opment of sc-models of the following subject domains[4] and their corresponding ontologies (in SCn):
Subject domain of project activities=> particular subject domain*:
Subject domain of actions of knowledge basessc-models developers=> particular subject domain*:
Subject domain of actions of problem solverssc-models developers
<= particular subject domain*:Subject domain of actions and tasks
Subject domain of problem fragments of knowledgebases=> particular subject domain*:
• Subject domain of incorrect fragments ofknowledge bases
• Subject domain of incompleteness in theknowledge base
• Subject domain of information garbage
All the listed sc-models of subject domains and corre-sponding ontologies are included in the relevant sectionsof the knowledge base of IMS.ostis Metasystems.
Next, we will consider in more detail the library ofreusable components of ostis-systems, the ostis-systemsdevelopment methodology, based on the family of subjectdomains and ontologies listed above, as well as thecorresponding automation tools.
VII. LIBRARY OF REUSABLE COMPONENTS OFSEMANTIC COMPUTER SYSTEMS
Reuse of ready-made components is widely used inmany industries related to the design of various typesof systems, because it allows to reduce the complexityof development and its cost (by minimizing the amountof labor due to the absence of the need to developany component), to improve the quality of the createdcontent. The use of ready-made components assumes thatthe distributed component is verified and documented,and possible errors and limitations are eliminated orspecified and known.
The basis for the implementation of the componentapproach within the OSTIS Technology is Library ofreusable components of ostis-systems, which is part ofthe IMS.ostis Metasystem.
It is important to note that since the library is part ofthe IMS.ostis metasystem, its replenishment is carried outin the same way as any other section of the knowledgebase in accordance with the knowledge base editingmechanism discussed below.
General structure of Library of reusable componentsof OSTIS, presented in SCn-code:
Library of reusable components of OSTIS= Library of OSTIS
62
= reusable components of OSTIS= reusable components of intelligent systems, build on
OSTIS Technology<= subdividing*:• Family of sc-models of computer systems
interpretation platforms• Library of reusable components of knowledge
bases sc-models• Library of standard components templates of
computer systems sc-models• Library of reusable components of problem
solver sc-models• Library of reusable components of user
interfaces sc-models• Library of typical subsystems of computer
systems developed by OSTIS Technology
Library of reusable components of OSTIS includes:
• set of such components;• means of the specification of such components;• tools of components search automation based on
their specifications.
The reusable component of OSTIS is generally un-derstood as a component of some ostis-system that canbe used in another ostis-system. For this, at least twoconditions must be met:
• it is technically possible to embed a component intoa child ostis-system by either physically copying,transferring and embedding it into the designedsystem, or using a component hosted in the originalsystem like a service, that is, without explicitlycopying and transferring the component. The com-plexity of embedding depends, among other things,on the implementation of the component;
• use of the component in any ostis-systems, exceptfor IMS.ostis Metasystem, is expedient, that is, acomponent cannot be a particular solution orientedto a narrow circle of tasks. It is worth noting,however, that in the general case almost everysolution can be used in any other systems, the rangeof which is determined by the degree of generalityand the domain dependence of such a solution.
From a formal point of view, each reusable componentof OSTIS is a structure that contains all those (and onlythose) sc-elements that are necessary for the componentto function in the child ostis-system and, accordingly,must be copied into it while including a componentinto one of such systems. The specific composition ofthis structure depends on the type of component andis specified for each type separately. In essence, thisstructure is a standard (or sample), which is copied whenthe corresponding component is included in the childsystem.
Each reusable component of OSTIS can be atomicor non-atomic, that is, it can consist of simpler self-contained components.
At any given time in the current state of the sc-memory, each reusable component can be fully repre-sented, that is, all sc-arcs of membership that connectthe structure corresponding to the component and allits elements are clearly present in memory, or it canbe presented implicitly, for example, by setting thedecomposition of this component into more particularones.
A. Library of reusable knowledge base components
The main semantic classes of reusable components ofknowledge bases stored in the library of components ofknowledge bases include:
The main semantic classes of reusable components ofknowledge bases stored in the library of components ofknowledge bases include:
• semantic neighborhoods of various entities;• ontologies of various subject domains;• specifications of formal languages describing vari-
ous subject domains;• sections of the knowledge base of various semantic
types (including non-atomic ones);• knowledge base of entire subsystems that provide
solutions to various problems;• knowledge bases of applied system;• and others.
Each reusable knowledge base component must bespecified within the library. This specification includesthe following minimum required information:
• information about the authorship of the component,i.e. the connection of the component with the signof the author (individual, team, etc.);
• information about the atomicity or non-atomicity ofthe component;
• information about the semantic class of a compo-nent by specifying that the component belongs to aclass of reusable components;
• information about the subject domain, a fragmentof which is described in the component;
• description of the purpose of the component, itsfeatures;
• date the component was created and last modified;• information about the dependent components for
this, that is, such components that cannot be usedseparately from this. An example of such compo-nents are the components describing the ontologiesof the subject domain of persons and the subjectdomain of historical personalities, since the firstcontains specifications of the concepts used to de-scribe the objects of the second subject domain;
63
• information about the openness of the componentand the possibilities of its use in various systemsfrom the point of view of proprietary.
This list can be expanded if necessary.An example of the specification of a reusable knowl-
edge base component is shown in the figure 2.Integration of the reusable component of the knowl-
edge base into the system is reduced to merging of keynodes by identifiers and eliminating possible duplicationsand contradictions that could arise if the developer ofthe child system manually made any changes to itsknowledge base.
To ensure the semantic compatibility of componentsof knowledge bases, it is necessary to:• match the semantics of all used key nodes;• match the main identifiers of the key nodes used
in different components. After that, the integrationof all components that make up the library in anycombination is carried out automatically, withoutintervention by the developer, using the mechanismsproposed in [75].
Components automation tools include the followingsc-agents:• agent of formation of a non-atomic component from
the atomic components – the task of this agent isto explicitly form a structure containing all the sc-elements that make up the indicated non-atomiccomponent;
• agent of dependency searching between components– the task of this agent is to search for all com-ponents, without which the use of the specifiedcomponent is impossible. In this case, the searchis performed recursively, taking into account thedependence of other components;
• agent of search for all non-atomic components,which include the indicated component;
• agent of component search, within which the indi-cated concepts are described;
• agent of component search by specification frag-ment.
The most important component of the library isKnowledge base kernel, which is the basis for buildingthe knowledge base of any system, since it contains theset of the top-level ontologies:• Subject domain of sc-elements;• Subject domain of entities;• Subject domain of sets;• Subject domain of structures;• Subject domain of knowledge;• Subject domain of semantic neighborhoods;• Subject domain of subject domains;• Subject domain of ontologies.On the basis of the proposed knowledge base model,
ontologies of subject domains describing the types of
knowledge that are used in most intelligent systems weredeveloped:• Subject domain of situations and events in sc-
memory;• Subject domain of relations and connections;• Subject domain of parameters and values;• Subject domain of logical formulas and logical
ontologies;• Subject domain of unified logical-semantic models
of computer systems;• Subject domain of numbers and number structures;• Subject domain of actions and tasks;• Subject domain of information constructions that do
not belong to the SC-code;• Subject domain of temporary entities;• Subject domain of characters that are not elements
of SC-code texts;• and others.Together with the Knowledge base kernel, the ontolo-
gies of the specified subject domain make up Extendedknowledge base kernel.
B. Library of reusable components of problem solvers
Classification of reusable components of problemsolvers in SCn-code:
Library of reusable components of problem solvers= reusable component of problem solvers<= subdividing*:• Library of reusable problem solvers• Library of reusable atomic abstract sc-agents• Library of reusable sc-text processing programs
The reusable abstract sc-agent means the componentcorresponding to a certain abstract sc-agent that can beused in other systems, possibly as part of more complexnon-atomic abstract sc-agents. The specified abstract sc-agent is included in the corresponding structure underthe key sc-element’ attribute. Each reusable abstract sc-agent must contain all the information necessary forthe operation of the corresponding sc-agent in the childsystem.
Classification of reusable sc-agents in SCn-code:
Library of reusable abstract sc-agents= reusable abstract sc-agent<= subdividing*:• Library of information search sc-agents• Library of sc-agents of immersing integrable
knowledge in the knowledge base• Library of sc-agents for align ontology of
integrable knowledge with the basic ontology ofthe current state of the knowledge base
64
Figure 2. Example of the specification of a reusable knowledge base component
• Library of sc-agents for planning explicitlydefined tasks
• Library of sc-agents of logical inference• Library of sc-models of high-level programming
languages and their corresponding interpreters• Library of sc-agents of knowledge base
verification• Library of sc-agents of knowledge base editing• Library of sc-agents for knowledge developers
activity automating
For the convenience of working with the library ofreusable components, tools for automating the search
for components based on a given specification have alsobeen developed, which are implemented as non-atomicsc-agent, which is decomposed into particular ones.
The following is the structure of such an agent in theSCn-code:
Automation tools of library of reusable abstractsc-agents<= decomposition of sc-agent*:• Abstract sc-agent of forming a non-atomic
component from atomic components• Abstract sc-agent of search for all non-atomic
components of which a given atomic component65
is part• Abstract sc-agent search for all related
components• Abstract sc-agent of sc-agent search by initiation
condition• Abstract sc-agent of sc-agent search by the
result of work• Abstract sc-agent of scp-program search by
input/output parameters• Abstract sc-agent of sc-agents search for which
the elements of a given set are the keysc-elements
The non-atomic component of problem solvers is un-
derstood as such a component in which it is possible toselect other components that can be used independently,separately from the source component. Most often, non-atomic sc-agents act as such non-atomic components, aspart of which can be isolated self-sufficient sc-agentsthat can be used separately from the original non-atomic,or scp-program that are common to several agents andcan be used not only as part of a non-atomic sc-agent.Thus, the task of the Abstract sc-agent of forming anon-atomic component from the atomic is the formationof a structure containing the complete sc-text of thenon-atomic component, including the specifications ofall sc-agents in its composition, as well as the texts ofall the necessary scp-programs. The formation of such astructure is necessary in order to simplify the process ofcopying the specified component to other ostis-systems.
The related component is a component that is oftenused in the ostis-system simultaneously with some othercomponent. Such a relationship between components isspecified explicitly with the help of the related compo-nent* relation. Examples of such components are somesc-agent and a user interface command that allows theuser to initiate the execution of the specified agent withthe given arguments. In this case, the sc-agent willfunction even without the presence of such a commandin the system, however, to initiate it, it will be necessaryto form the corresponding structure in the sc-memorymanually.
Abstract sc-agent of sc-agents search for which theelements of a given set are the key sc-elements plays animportant role when making changes in the knowledgebase, in particular, when redefining any concepts. Thespecified sc-agent allows to identify those sc-agents thatmay require changes in the algorithm of work due tochanges in the semantic interpretation of any concepts.
VIII. METHODS OF SEMANTIC COMPUTER SYSTEMSDEVELOPMENT
As mentioned earlier, the development of the ostis-system comes down to the development of its knowledgebase. In this section, we will take a closer look at the
principles for the development of knowledge bases ofostis-systems, as well as some features specific to thedevelopment of ostis-system problem solvers.
In accordance with the requirements formulated above,the approach to the development of ostis-systems itselfdoes not limit the use of any specific methods forthe coordination of project activities. However, takinginto account the current goals of OSTIS Technology, inparticular, the need to develop the IMS.ostis Metasys-tem, currently the development methodology of ostis-systems and the corresponding tools implement an open-source project coordination mechanism borrowing themain ideas from• traditional modern tools of collective development
of such projects, for example [76]• modern approach to the review of scientific articles
adopted in the vast majority of scientific journals.The most important features of the proposed method-
ology and tools are ensuring at each time point theintegrity and consistency of the current version of theknowledge base directly during its operation, as wellas the transparency and openness of the agreementmechanism, which are achieved through the previouslypresented principles.
The proposed method involves two main stages – thestage of creating the initial version of the developed ostis-system, whose knowledge base is synthesized from thecomponents of the library of the reusable componentsof the ostis-systems knowledge base, and the stage ofexpanding and improving the knowledge base of theostis-system being developed. The initial version of theostis-system contains a set of knowledge and tools forproblems solving, sufficient for the further developmentof the system.
The process of creating the starting version of theostis-system can be divided into next main stages:• selection and installation of ostis-systems sc-models
interpreting platform;• installation of Knowledge bases kernel from the
library of reusable components of knowledge bases;• installation of Problem solver kernel from the
library of reusable components of problemsolvers [5], that is, a set of basic reusablecomponents of problem solvers necessary for thestarting version of the ostis-system;
• installation of Interface kernel [77], i.e. a set of ba-sic reusable components of sc-models of interfacesnecessary for operation of the starting version ofthe ostis-system;
• installation of support system for collective devel-opment of knowledge bases.
After the basic configuration of the starting version ofthe ostis-system is assembled, the stage of developingthe knowledge base begins, which is discussed in moredetail below.
66
A. Structuring knowledge base from the point of view ofthe development process
To support the evolution of the ostis-system, it isnecessary to distinguish sections of the knowledge basecontaining information on its development plans (thefuture of the system), current development processes,including the current processes for coordinating changesto the knowledge base, as well as information on com-pleted knowledge base development processes in orderto provide the ability to track and cancel changes to theknowledge base.
Thus, from the point of view of the developmentprocess, the knowledge base is conventionally dividedinto overlapping areas, describing the part of the knowl-edge base that is available for operation to the end user(agreed part of the knowledge base), a section containinginformation about the operation of the system, and asection containing information about the evolution of thesystem.
Figure 3 shows the structure of the knowledge basefrom the point of view of the development process.
The agreed part of the knowledge base is the partof the knowledge base that is agreed between all theparticipants in the development at the current time. Theknowledge presented in this section of the knowledgebase is available to the end users of the system in theoperating mode of the system. The distinguishing of theagreed part of the knowledge base is necessary in order tobe able to hide from the end user system information thatis not directly related to the operation of an intelligentsystem.
In turn, agreed part of the knowledge base is dividedinto subject part of the knowledge base, context of thesubject part of the knowledge base within the GlobalKnowledge Base and computer system documentation(figure 3).
By Global knowledge base we mean the global ab-stract semantic space of all knowledge accumulated bymankind to the current time [2].
Subject part of the knowledge base contains all infor-mation about the subject domain (or several interrelatedsubject domains within the same knowledge base) forwhich the developed system is intended to work. Forexample, the section describing Euclidean geometry inthe geometry intelligent reference system.
The context of the subject part of the knowledge basewithin the Global knowledge base contains a specifica-tion of objects that are not directly studied in the subjectpart of the knowledge base of this system, but are relatedto it, i.e. used for description of concepts studied in thesubject part of the knowledge base. For example, for theIMS.ostis Metasystem, these could be such concepts asartificial intelligence or intelligent system, for the systemaccording to Euclidean geometry - historical informationabout Euclidean life, mathematics, etc.
The computer system documentation section containsdocumentation of the ostis-system itself, at a minimum,the specification of its knowledge base, problem solverand interface, as well as all the necessary manuals thatprovide the opportunity for learning in working with thesystem.
Section history and current processes of computersystem operation includes the following sections:• computer system operation history;• current processes of computer system operation.The computer system operation history section stores
the history of the system’s dialogue with its users.The current processes of computer system opera-
tion stores the specifications of all actions performedby the ostis-system at the moment (which are presententities), as well as all temporary auxiliary constructions,generated by sc-agents in the process of work and notyet deleted. After performing these actions, their signsand specifications are transferred to the computer systemoperation history section.
The history, current processes and development planof computer system section is decomposed into thefollowing sections:• structure and organization of a computer system
project;• history of the computer system development;• current development processes of computer system;• computer system development plan.Section structure and organization of a computer
system project describes the structure of a project aimedat the ostis-system development, including its subprojectsand the roles of the developers responsible for eachproject.
The history of computer system development sectioncontains specifications of project activities performedduring system development (past entities), with the oblig-atory indication of the performers, the sequence and theresult of each activity. The presence in the knowledgebase of this kind of information will allow to rollbackthe changes made to the knowledge base, as well as totake into account completed design tasks when planningfurther work within the project.
The current development processes of computersystem section contains specifications of approved andinitiated project actions performed by the system devel-opers at a given time (real entities), with the obligatoryindication of the performers, the sequence and purposeof the implementation, and also all the informationdescribing the proposals for editing the subject part of theknowledge base and computer system operation historyand their discussion by administrators, managers andexperts.
In the computer system development plan sectionthere are specifications of project actions that are ap-proved for execution, but have not yet been fulfilled for
67
Figure 3. The structure of the knowledge base from the point of view of the development process
any reason, as well as all the information describingproposals for editing the section history, current pro-cesses and development plan of computer system andtheir discussion by administrators, managers and experts.
B. General mechanism for knowledge bases sc-modelsdevelopment
The process of a knowledge base development is asequence of the following steps:
• Formation of the initial structure of the knowledgebase, which involves:•• the formation of the structure of the knowl-
edge base sections corresponding to the abovementioned variant of structuring the knowledgebase from the point of view of the developmentprocess;
•• identification of the described subject domains;•• building a hierarchical system of the described
subject domains;•• building a hierarchy of knowledge base sections
within the subject part of the knowledge base,which takes into account the hierarchy of sub-ject domains constructed at the previous stage.
• Identifying knowledge base components that can betaken from a library of reusable knowledge basecomponents and including them into the knowledgebase that is being developed.
• Formation of project tasks for the development ofmissing fragments of the knowledge base and theassignment of tasks to developers.
• Development and coordination of knowledge basefragments, which, in turn, may later be included inthe library of reusable knowledge base components.
• Verification and debugging of the knowledge base.
It should be noted that in the process of the knowledgebase improving, stages 3–5 are performed cyclically.
Figure 4 shows a diagram reflecting the sequence of aknowledge base building steps according to the proposedmethodology.
The basis of the methodology under consideration isa formal model of developer activity aimed at develop-ing and modifying of knowledge bases, formal meansfor specifying proposals for editing a knowledge base,method for introducing changes to the knowledge base,formal means for specifying transition processes in theknowledge base, and formal means for specifying con-tradictions and incompleteness in the knowledge base.
To ensure the reflexivity of the intelligent system,in particular, the ability to automate the analysis ofthe history of the evolution of the knowledge base andgenerate plans for its development, all activities relatedto the development of the knowledge base are specifiedin this knowledge base by the same means as the subjectpart.
The process of creating and editing the knowledgebase of the ostis-system is reduced to the formationof proposals for editing of a particular section of theknowledge base by developers (picture 5) and the sub-sequent consideration of these proposals by knowledgebase administrators. In addition, it is assumed that, ifnecessary, experts can be involved in verifying incomingproposals for editing the knowledge base, and the de-velopment process is managed by managers of relevantknowledge base development projects. In this case, theformation of design tasks and their specification are alsocarried out using the mechanism of proposals for editingthe relevant section of the knowledge base. Thus, allinformation related to the current processes of developinga knowledge base, history and plans for its developmentis stored in the same knowledge base as its subject part,that is, the part of the knowledge base accessible to theend user of the system. This approach provides wideopportunities to automate the process of knowledge bases
68
Formation of the initial structure ofthe knowledge base
Identification of the components ofthe knowledge base, which can be
borrowed from the library
Formation of project assignments forthe development of missing fragments
of knowledge base
Development and coordination offragments of knowledge base
Verification and debugging ofknowledge base
Semantic model of knowledge base
Library of reusable components of knowledge base
Activity model ofknowlege base developers
Means of proposal specification
The way to make changes in the knowledge base
Means of specification of transitional processes of evolution of knowledge base
Means of specification of contradictions and incompleteness in the knowledgw base
Formation of the structure ofknowledge base sections
Identify the described subjectdomains
Building a hierarchicalsystem of the described
subject domains
Construction of the sectionshierarchy of the knowledgebase in the subject part of the
knowledge base
Formation of the starting versionof ostissystem
Figure 4. Methods of building and modifying of knowledge bases
creation, as well as subsequent analysis and improvementof the knowledge base.
Figure 5. Illustration of the knowledge base editing process
Each proposal for editing the knowledge base is astructure containing sc-text that is proposed to be in-cluded in the agreed part of the knowledge base. Thestructure of such proposals may include signs of actionsfor editing the knowledge base, which are automaticallyinitiated and executed by the relevant agents after theapproval of the proposal.
Figure 6 shows the stages of developing a certain pieceof the knowledge base, starting with the formulation ofthe project task and ending with the final approval orrejection of the proposal for editing the knowledge base.
The proposed methodology for developing a knowl-edge base is primarily focused on open-source projects
for developing intelligent systems, where anyone canbecome a developer.
Next, we consider in detail the typology of developerroles and classes of actions they perform.
C. Typology of knowledge base developers
First of all, all users of any ostis-system are dividedinto registered users and unregistered users.
To describe this fact, the following relations are usedin the knowledge base:
• unregistered user is a binary relation connectingostis-system and sc-element, denoting person thatdid not pass the registration procedure in system;
• registered user is a binary relation connecting ostis-system and sc-element, denoting person that haspassed the registration procedure in system.
An unregistered user has access to read the subject partof the ostis-system knowledge base. This type of userscan work with the ostis-system in the operation mode,i.e. it can only set queries addressed to the subject partof the knowledge base (i.e., solve subject problems).
A registered user has access to read the entire knowl-edge base and make proposals to the entire knowledgebase, can play the role of the end user of the ostis-system,that is, work in the operating mode, and also the role ofits developer. At the same time, regardless of the rolethat a particular user performs, he can make proposals forediting any part of the knowledge base, which, depending
69
Figure 6. Illustration of the mechanism of making changes in the knowledge base
on its level, will either be automatically accepted orseparately considered.
Among registered users there is a separate type ofusers - developer.
Developer is a binary relation connecting a project todevelop a section of the knowledge base of the ostis-system (in the limit, the entire knowledge base) and asc-element denoting a person who can be the developerof this section of the knowledge base, i.e. perform projecttasks within this section.
In addition to operating the ostis-system, the devel-oper can make proposals for changing any part of theknowledge base, leave comments on the such propos-als. Among the developers, such roles as administrator,manager and expert are distinguished.
Administrator is a binary relation connecting a projectto develop the knowledge base section of the ostis-system (in general, the entire knowledge base) and thesc-element denoting the person who is the administratorof this knowledge base section.
Tasks of administrator are:• control of the integrity and consistency of the entire
knowledge base;• define access levels for other users;• a decision regarding the acceptance or rejection of
proposals in various parts of the knowledge base,including, if necessary, sending them for expertise;
• making changes in various parts of the knowledgebase by using the appropriate editing commands(in this case, changes are automatically made outas proposals and entered into the section of thedevelopment history of the ostis-system).
If it is necessary to develop a knowledge base of abig size, a hierarchy of developers can be introducedcorresponding to the hierarchy of sections of the knowl-edge base being developed. In this case, the approvalof a proposal by the administrator of the lower levelsection does not lead to the integration of the proposalinto the appropriate section, but requires considerationby the higher level administrators. The final decision ismade by the administrator of the entire knowledge base.Figure 7 shows a fragment of the knowledge base thatdescribes the hierarchy of knowledge administrators.
Manager is a binary relation connecting a projectto develop the knowledge base section of the ostis-system (in general, the entire knowledge base) and thesc-element denoting the person who is the manager ofthis knowledge base section.
Tasks of the manager are:• planning the amount of work on the development
of the knowledge base;• detailed elaboration of project tasks for subtasks,
formulation of project tasks, assignment of perform-ers of project tasks;
70
Figure 7. Knowledge base administrators hierarchy
• setting priorities and deadlines for tasks completing;• control the timing of project tasks.Manager makes changes to the part of the relevant sec-
tion that describes the project tasks using the appropriateediting commands, and the changes are automaticallypresented as proposals and entered into the section de-scribing the development plan of the ostis-system. Thus,the manager is the administrator of the specified section.
Expert is a binary relation connecting a project todevelop a section of the knowledge base of the ostis-system (in general, the entire knowledge base) and ansc-element denoting a person who is an expert of thisknowledge base section.
Tasks of the expert are:• verification of the results of the project tasks;• if necessary, the expert can leave comments on
any fragment of the knowledge base regarding itscorrectness. All comments fall into the section de-scribing the plan for the development of a computersystem.
In addition, any participant in the development process
has the opportunity to leave a natural language com-mentary to any fragment or element of the knowledgebase, thus, any issues related to the specified fragmentor element of the knowledge base can be discussed. Suchcomments fall into the knowledge base section currentprocesses of computer system development.
An example of using these relations to indicate theroles of developers in a project to create a knowledgebase in the SCg is presented in the figure 8:
D. Ontology of actions of knowledge base developers
In the process of developing the knowledge baseof the ostis-system, each of the users involved in thedevelopment process uses a specific set of commandscorresponding to the knowledge base editing mechanismdescribed above. Each such command corresponds to acertain class actions in sc-memory [5]. All such actionsare combined into a common class actions of the knowl-edge base developer.
For the purpose of subsequent automation, someclasses of actions of the knowledge base developer are
71
Figure 8. The roles of developers in the knowledge base
formally specified and detailed to the level of elementarytransformations in the system memory.
To identify classes of actions, the names of the fol-lowing form will be used: action. «action class name».
The hierarchy of the actions of the knowledge basessc-models developers in the SCn-code, taking into ac-count the roles considered and the corresponding re-sponsibilities (the actions of the manager are not shown,since they repeat the actions of the administrator, but areapplicable only for certain sections):
knowledge bases sc-models developer action⊃ action of knowledge base ordinary developer⊃ action. build a new fragment for inclusion in the
knowledge base⊃ action. modify the proposal for editing the
knowledge base⊃ action. make a proposal for editing the knowledge
base⊃ action. form a project task proposal⊃ action. form a project task performer proposal
⊃ action of knowledge base administrator⊃ action. consider a proposal for editing the
knowledge base⊃ action. approve a proposal for editing the
knowledge base⊃ action. reject a proposal for editing the knowledge
base⊃ action. create a task for the verification of the
proposal
⊃ action. approve the result of the proposalverification
⊃ action. reject the result of the proposal verification⊃ action of knowledge base manager⊃ action of knowledge base expert⊃ action. verify the specified structure⊃ action. approve verifying proposal⊃ action. reject verifying proposal⊃ action. create a task for consideration of the
proposal verification result
For the specification of knowledge bases sc-models de-veloper action and structures describing the proposal forediting the knowledge base, relations such as proposal*,approved*, rejected*, new version*.
An example of a specification of a proposal for editinga knowledge base using the above relations and classesof actions in the SCg-code is presented in the figure 9:
E. Typical mistakes and difficulties in the developmentof knowledge bases of semantic computer systems
As mentioned earlier, one of the important tasks ofIMS.ostis Metasystem is the information support for thedevelopers of sc-models of ostis-systems, which also in-volves learning of developers using typical examples andexercises. To solve this problem, a section of the knowl-edge base of the Metasystem IMS.ostis was developed,describing typical errors and difficulties in developing ofknowledge bases sc-models. In this section we considersome of the most common examples of such problems.
1) It is necessary to distinguish:• Syntactic typology of sc-elements (sc-node, sc-
edge, sc-arc);• Semantic typology of sc-elements.That is, membership pair 6= sc-arc of membership
membership pair⊃ sc-arc of membership
2) It is necessary to distinguish:• variable sc-arc of membership;• constant sc-arc of membership.Sometimes, for example, constant sc-arcs of member-
ship can go out of constant sc-nodes, but are included invariables sc-elements (figure 10).
3) It is necessary to distinguish:• permanent (stationary) sc-arcs of membership;• temporal (situative) sc-arcs of membership.4) It is necessary to distinguish:• the entity being described;• abstract sign (internal sign, sc-sign) of the described
entity;• external sign (identifier, designation, name) of the
described entity;• specific occurrence of an external sign of the de-
scribed entity in a specific sign structure;72
Figure 9. Specification of the proposal for editing the knowledge base
Figure 10. Incidence of constant sc-arcs and variable sc-nodes
• unambiguous specification of the described entity,represented in sc-memory or in any external lan-guage.
5) It is necessary to distinguish:• set of sc-elements itself, which can be a described
entity;• sc-element, which is the sign of the corresponding
(denoted by it) set of sc-elements;• sc-text, which describes the connection of the sign
of some set of sc-elements with all sc-elements thatare members of this set.
6) It is necessary to distinguish:• concept;• natural language text (text file), which is one of the
wording of the concept definition;• natural language text that is an another formulation
of the concept definition;
• text, which is the natural language formulation of astatement, which could be a concept definition, butis not (a statement of a defining type);
• sc-node denoting a statement, which is the conceptdefinition, presented in the SC-code;
• sc-node denoting the entire sc-construction, whichis the concept definition, presented in the SC-code.
7) It is necessary to distinguish:• concept of set;• concept of a set of sc-elements (semantically nor-
malized set, sc-set).8) It is necessary to distinguish:• sc-sign atomic logical formula;• sc-sign non-atomic logical formula;• sc-sign of the full sc-text of the logical formula.
For an atomic logical formula, the sc-sign of thisformula coincides with the sign of its full text.
9) It is necessary to distinguish:• concrete number as a sign of the corresponding
abstract entity;• unambiguous specification of this number, for ex-
ample, in one or another number system;• string of digits, which is the external identifier
(name) of this number corresponding to a particularnumber system.
10) It is necessary to distinguish:• constant sc-element that is a sign of a specific
73
known (identified), uniquely defined, specified en-tity – known sc-constant;
• variable sc-element that is a sign of an arbitraryentity from some additionally defined (using logicalstatements) set of entities – sc-variable;
• constant sc-element, which is a sign of a specificentity, but not currently known – unknown sc-constant.
11) It is necessary to distinguish:• sc-constant which is a member of a given set;• sc-variable, any value of which is a member of a
given set;• sc-variable, which itself is a member of a given set
(for example, a structure).12) It is necessary to distinguish:• sign of some specific (constant) entity class. At the
same time, the elements (instances) of a class canbe signs of other classes, signs of variables, signsof specific temporary entities, and signs of specificpermanent entities;
• sign of some constant element (instance) of theabove class;
• sign of a specific subset (subclass) of the specifiedclass;
• sign of some arbitrary (variable) entity, possiblevalues of which can only be signs of elements ofthe considered class of entities;
• sign of some arbitrary (variable) entity, one of thevalues of which is the sign of the entity class itself.
13) It is necessary to distinguish a section describingthe subject domain and the subject domain beingdescribed
14) It should be remembered that for binary orientedrelations there is no semantic need to introduceinverse relations, i.e. semantically, all links of eachbinary relation are also links and its inverse relationand vice versa (sin*=arcsin*, be a subset*=be asuperset*)
15) It is necessary to distinguish:• sign of non-role relation;• sign of role relation corresponding to a given non-
role relation;• signs of relation domains (a domain is a set of those
and only those entities that, in the tuples of a givenrelation, perform the specified role).
16) It is necessary to distinguish the connection ofmembership and inclusion* (figure 11).
17) It is necessary to distinguish:• case when the element ei is included in the set si,
while performing multiple roles at the same time(figure 12)
• case when the element ei is included in the setsi multiple times. Moreover, within the framework
Figure 11. Membership and inclusion
Figure 12. Multiple element roles in the set
of different occurrences, the specified element mayperform different roles (figure 13)
Figure 13. Multiple occurrences of an element in a set with differentroles
18) It is necessary to distinguish the sc-element, de-noting some described entity and a singleton, theonly elements of which is the sign of this entity(figure 14)
19) It is necessary to distinguish the relation betweenclasses and the relation between instances of theseclasses (figures 15, 16)
F. Methods for sc-models of problem solvers develop-ment
The proposed methods for problem solvers construct-ing and modifying includes several stages. Figure 17presents a list of such stages, indicating the sequenceof their execution.
The considered methods can be applied both in thedevelopment of hybrid solvers and in the developmentof simpler solvers, since from a formal point of view allof them are treated as a non-atomic abstract sc-agent.
74
Figure 14. The difference of the sign of the entity and the singletoncontaining this sign
Figure 15. Incorrect (left) and correct (right) example of using theconcepts of a triangle and a segment
Stage1. Requirements formation and problemsolver specification
At this stage, it is necessary to clearly identify theproblems that should be solved by the problem solver,consider the intended ways of solving them and, basedon this analysis, determine the place of the future solverin the general hierarchy of solvers. The importance ofthis stage lies in the fact that, with proper classification,
Figure 16. Incorrect (left) and correct (right) example of using theconcept of intersection*
Figure 17. Stages of the process of problem solvers constructing andmodifying
there is a possibility that there is already an implementedversion of the required solver in the component library.Otherwise, however, the developer has the opportunity toinclude the developed solver into the component libraryfor later use. These facts are due to the fact that thestructure of the library of problems solvers componentsis based on the semantic classification of such solversand, accordingly, of their components.
With an insufficiently precise specification and classi-fication of the solver being developed, it is more likelythat a suitable solver will not be found in the componentlibrary, even if it is there, and the newly developedsolver cannot be included in the library. Thus, the idea ofreuse of already developed components will be broken,which will significantly increase the cost of such a solverdevelopment.
Stage2. Formation of a sc-agents collective that arepart of the developed solver
In the case when it is not possible to find a ready-made solver in the library that meets all requirements, itis necessary to distinguish and specify all the componentsof such a solver.
The result of this stage is a list of fully specifiedsc-agents, which will be part of the developed solver,with their hierarchy up to atomic sc-agents. Within thisstage, it is very important to design a group of agentsin such a way as to maximize the use of reusablecomponents already presented in the library, and incase of the necessary component absence, be able toinclude it in the library after implementation. Dependingon the complexity of the solver being developed, suchcomponents can include both atomic sc-agents and whole
75
teams of sc-agents (non-atomic sc-agents).When developing the list of agents (including their
specifications) it is necessary to follow a number ofprinciples:
• each sc-agent being developed should be as inde-pendent as possible, that is, the set of key nodesof this sc-agent should not include concepts thatare directly related to the subject domain underconsideration. The exceptions are concepts fromgeneral subject domains that are interdisciplinaryin nature (for example, the inclusion* relation orthe action concept). This rule can also be violatedif the sc-agent is auxiliary and is focused on pro-cessing a particular class of objects (for example,sc-agents that perform arithmetic calculations canwork directly with specific relations addition* andmultiplication*, etc.). All the sc-agent informationneeded to solve the problem must be extracted fromthe semantic neighborhood of the correspondinginitiated action. Obviously, the sc-agent, developedwith these requirements, can be used to design alarger number of osti-systems than if it was imple-mented with a focus on a particular subject domain.After development and debugging is completed,such a sc-agent should be included in the Libraryof reusable abstract sc-agents;
• it is important to distinguish the concept of sc-agentand agent program (including agent scp-program).The interaction of sc-agents is carried out exclu-sively through the specification of information pro-cesses in common memory, each sc-agent respondsto a certain class of events in sc-memory. Thus, eachsc-agent corresponds to a condition of initiation andone agent program that starts automatically when anappropriate condition of initiation occurs in the sc-memory. In this context, various subprograms canbe called as many times as necessary. However,it is important to distinguish the initiation of thesc-agent, which occurs when the correspondingconstruction appears in the sc-memory, and thesubprogram call by another program, which impliesan explicit specifying of the called subprogram andthe list of its parameters;
• each sc-agent should independently verify the com-pleteness of its own initiating condition in thecurrent state of sc-memory. In the process of prob-lem solving, a situation may arise when severalsc-agents reacted to the appearance of the samestructure. In this case, the execution continues onlythose of them, the condition of initiation of whichis fully consistent with the situation. The remainingsc-agents in this case stop execution and returnto the standby mode. The implementation of thisprinciple is achieved by carefully specifying thespecifications of the developed sc-agents. In the
general case, the initiation conditions for several sc-agents may coincide, for example, in the case whenthe same task can be solved in different ways andit is not known in advance which of them will leadto the desired result;
• it is necessary to remember that a non-atomic sc-agent from the point of view of other sc-agents thatare not part of it must function as an integral sc-agent (perform logically atomic actions), which im-poses certain requirements on the specifications ofthe atomic sc-agents included in its composition: asa minimum, it is necessary that at least one atomicsc-agent is present in the composition of a non-atomic sc-agent, the initiation condition of whichcompletely coincides with the initiation conditionof this non-atomic sc-agent;
• if necessary, the implementation of a new sc-agentshould be guided by the following principles foratomic abstract sc-agents design:
•• the designed sc-agent should be as independentas possible from the subject domain, which willenable it to be used in the development ofsolvers for the maximum possible number ofosti-systems in the future. At the same time,universality implies not only minimizing thenumber of key nodes of the sc-agent, but alsodistinguishing the class of actions performedby this sc-agent in such a way that it makessense to include this sc-agent into the Libraryof reusable abstract sc-agents and use it whendeveloping solvers of other ostis-systems. Oneshould not artificially link a set of actionsinto one sc-agent and, conversely, dismemberone self-sufficient action on sub-actions: thiswill cause difficulties in understanding how sc-agents work by developers and will not allowusing sc-agent in some systems (for example,in learning systems which should explain thedecision-making way to the user);
•• the act of activity of each sc-agent (the ac-tion performed by this sc-agent) must be log-ically consistent and complete. It should beremembered that all sc-agents interact exclu-sively through common memory and avoid sit-uations in which the initiation of one sc-agentis performed by explicitly generating a knowninitiation condition by another sc-agent (i.e., infact, explicitly direct calling one sc-agent byanother);
•• it makes sense to separate into sc-agents thoserelatively large fragments of the implementationof a certain general algorithm that can be exe-cuted independently of each other;
• when combining sc-agents into teams, it is recom-mended to design them in such a way that they can
76
be used not only as part of the non-atomic abstractsc-agent considered. If this is not possible and somesc-agents, being separated from the team, lose theirmeaning, it is necessary to indicate this fact whendocumenting the sc-agents;
• the actual initiator of the sc-agent launch via com-mon memory (the author of the corresponding con-struction) can be either the system user directly oranother sc-agent, which should not be reflected inthe work of the sc-agent itself.
Stage3. Development of algorithms for atomic sc-agents
Within the framework of this stage, it is necessaryto think over the algorithm of each developed atomicsc-agent. The development of the algorithm implies thedistinguishing of logically consistent fragments in it,which can be implemented as separate scp-programs,including those executed in parallel. Thus, there is aneed to speak not only about the Library of reusableabstract sc-agents, but also about Library of reusableprograms for sc-texts processing in various programminglanguages, including Library of reusable scp-programs.Due to this, part of the scp-programs that implement thealgorithm of the operation of a certain sc-agent can betaken from the corresponding library.
It is important to remember that if sc-agent generatesany temporary structures in memory during the work,then at the completion of the work it is necessary todelete all information, the use of which in the systemis no longer advisable (to remove information garbage).The exceptions are situations when such information isnecessary for several sc-agents to solve one problem,but after solving a problem, the information becomesuseless or redundant and requires removal. In this case,a situation may arise when none of the sc-agents is ableto remove the garbage. In this case, there is a need totalk about the inclusion of specialized sc-agents into thesolver, whose task is to identify and destroy informationgarbage.
Stage4. Implementation of scp-programsThe final stage of development is the implementation
of previously specified scp-programs or, if necessary,programs implemented at the platform level.
Stage5. Verification of the developed componentsThe verification of the developed components can be
carried out both manually and using the specified toolsthat make up the automation system for constructing andmodifying of problem solvers built on the base of OSTISTechnology.
Stage 6. Debugging of developed components. Errorcorrection
The debugging phase of the developed components, inturn, can also be divided into more specific stages:• debugging of individual scp-programs or programs
implemented at the platform level;
• debugging of individual atomic sc-agents;• debugging of non-atomic sc-agents included in the
problem solver;• debugging of the entire problem solver.Note that Stage5 and Stage6 can be executed in
parallel and are repeated until the developed componentsmeet the necessary requirements.
G. Ontology of the activity of problem solvers developers
In the framework of the proposed approach, the meth-ods for constructing and modifying problem solvers isbased on the formal ontology of the activities of suchsolvers developers.
It is important to note that, according to the modelmentioned earlier, the problem solver is a abstract sc-agent, in connection with which the development of asolver comes down to the development of such an agent.
A fragment of a formal ontology of activity aimed atconstructing and modifying problem solvers in the SCn-code looks as follows (for convenience of reading, therelations defining the order of actions are omitted):
action. develop an osti-system problem solver= action. develop an abstract sc-agent<= subdividing*:• action. develop an atomic abstract sc-agent• action. develop non-atomic abstract sc-agent
=> abstract subaction*:• action. specify an abstract sc-agent• action. find an abstract sc-agent in the library
that satisfies the given specification• action. verify sc-agent• action. debug sc-agent
action. develop a platform-independent atomicabstract sc-agent=> abstract subaction*:• action. decompose a platform-independent
atomic abstract sc-agent into scp-programs• action. develop an scp-program
action. develop a non-atomic abstract sc-agent=> abstract subaction*:• action. decompose a non-atomic abstract
sc-agent into particular• action. develop an abstract sc-agent
action. develop an scp-program=> abstract subaction*:• action. specify scp-program• action. find in the library an scp-program that
satisfies the given specification• action. implement the specified scp-program• action. verify scp-program• action. debug scp-program
77
action. verify sc-agent<= subdividing*:
• action. verify atomic sc-agent• action. verify non-atomic sc-agent
action. debug sc-agent<= subdividing*:
• action. debug atomic sc-agent• action. debug non-atomic sc-agent
The presence of such a formal ontology allows, firstly,to partially automate the process of constructing andmodifying solvers, and secondly, to increase the effec-tiveness of information support for developers, sincethis ontology is included in the knowledge base of theIMS.ostis Metasystem.
IX. AUTOMATION TOOLS FOR THE DEVELOPMENT OFSEMANTIC COMPUTER SYSTEMS
A. Architecture of automation tools for the developmentof knowledge bases sc-models
To reduce the complexity of the process of developingknowledge bases and reduce the requirements for devel-opers in the framework of OSTIS Technology, tools havebeen developed to automate the processes of knowledgebases development and information support for suchknowledge bases developers.
The information support tools for developers are im-plemented in the form of the previously mentionedIMS.ostis Metasystems [73].
Tools for automating knowledge base developmentprocesses are implemented in the form of system forthe collective knowledge bases development support(SKBD). An important aspect of the knowledge basesdevelopment support is support of activities of knowl-edge base developers directly during the operation of thesystem being developed, which is possible due to the factthat the support system for the collective development ofknowledge bases is embedded as a subsystem into eachdeveloped system.
Figure 4 shows the stages of the process of devel-oping a knowledge base in accordance with the methodsdescribed above. Actions performed by developers in thefirst two stages cannot be fully formalized, and thereforetheir implementation cannot be fully automated.
Thus, in the framework of the system for the collectiveknowledge bases development support, the actions of de-velopers carried out in the last three stages are automated.These actions are performed cyclically throughout theentire life cycle of the system being developed (figure4).
The architecture system for the collective knowledgebases development support is presented in the figure 18.As can be seen from the figure, the system is an ostis-system and interacts with the IMS.ostis Metasystem,which includes a library of reusable components, whichallows, on the one hand, to take components availablein the library in accordance with the proposed methodsof knowledge bases developing, on the other hand – toprovide the opportunity to replenish the library with newcomponents obtained in the process of knowledge basesdevelopment.
Let us consider in more detail the composition of eachsystem component.
B. The knowledge base of system for the collectiveknowledge bases development support
The knowledge base of system for the collectiveknowledge bases development support includes sectionscontaining all the knowledge necessary to support theprocess of the knowledge base developing and evolving.
Such knowledge includes:
• set of top-level ontologies necessary for the func-tioning of the SKBD itself and being the basis forbuilding knowledge bases of the systems being de-veloped. These ontologies are part of the previouslyconsidered Knowledge base kernel;
• formal ontology of the subject domain of devel-opers’ activities aimed at knowledge bases devel-oping and modifying, including a description ofthe typology of roles of developers of knowledgebases, classification of developer actions, as wellas formal means of specifying proposals for editinga knowledge base. The concepts included in thisontology were discussed above;
• ontology of the subject domain of problem struc-tures in knowledge bases, i.e., those structures thatdescribe incomplete, incorrect or redundant infor-mation in the knowledge base;
• means for specifying changes and transients in theknowledge base.
1) Problem structures in knowledge bases and theirtypology: One of the tasks of the system for the collectiveknowledge bases development support is the identifica-tion of problem structures in the knowledge base withthe aim of correcting them.
Search and elimination of incompleteness, incorrect-ness and information garbage is carried out on the basisof:
• ontologies of completeness, which formally set therequirements for the completeness of the specifiedsubject domains in the sc-memory;
• ontologies, within which classes of constructionsare specified, representing incorrectness and infor-mational garbage in the respective subject domains.
78
UI IS
scmemory
KB IS Problem solver IS
Developed IS
System for the collective knowledge bases development support
Set oftoplevelontologies
Ontology of thesubject domain of
developers’ activitiesaimed at knowledgebases developing and
modifying
Ontology of thesubject domain of
problemstructures in
knowledge bases
Means forspecifyingchanges andtransients inthe KB
KB SKBD
KBdeveloper
A set of interfacecommands for the
engineer,administrator,manager andexpert of KB
UI SKBD
Nonatomic scagent of KB editing
Nonatomic scagent for automationof activities of ordinary KB developer
Nonatomic scagent for automationof activities of KB administrator
Nonatomic scagent for automationof activities of KB manager
Nonatomic scagent for automationof activities of KB expert
Nonatomic scagent for KBverification
Nonatomic scagent for knowledgebase characteristics calculating
Problem solver SKBD
IMS
Library of R
eusable Com
ponents of
know
ledge bases
Figure 18. Architecture of system for the collective knowledge bases development support
The selection of classes of problem structures in theknowledge base allows us to specify such structuresfor knowledge base developers and for their automaticprocessing by agents.
In addition, the specification of the problem structuresin the knowledge base allows the system to analyzeits own knowledge base for correctness, completenessand redundancy, evaluate acquired knowledge and skills,which ensures the property of reflexivity of the intellin-gent system.
Consider the typology of such structures:• incorrect structure;• structure describing the incompleteness in knowl-
edge base;• informational garbage.Under the incorrect structure we mean a structure
containing a fragment of the knowledge base, in whichin any way revealed any incorrectness. Additional con-cretization of the fact of incorrectness can be carried outby adding this structure to a particular class of incorrectstructures or by specifying additional relations specifyingthis structure, for example, the contradiction relation.
The structure describing the incompleteness in theknowledge base means a structure containing a fragmentof a knowledge base that lacks any information that isnecessary (or at least desirable) for an unambiguous andcomplete understanding of the meaning of this fragment.
By informational garbage is meant a structure contain-ing a fragment of the knowledge base, which for some
reason has become unnecessary and requires removal.The formation of a structure describing such a fragmentof the knowledge base, and, accordingly, the removal ofits contents can be performed both by the sc-agent thatgenerated the fragment and by the specialized sc-agentsof garbage collection.
The following are the selected classes of incorrectstructures:• structure that contradicts the property of uniqueness
(a special case of this class of structures is the classCantor set contains a repeating element);
• cycle within order relation – failure to comply withthe antisymmetry property for the order relation;
• mismatch of elements tuple with relationship do-mains;
• power mismatch of relation arity;• elements of a single subdividing have a non-empty
intersection;• mismatch of a fragment of the knowledge base with
a logical statement;• and more.As an example, consider the detection in the knowl-
edge base of the contradictions associated with the indi-cation of the angle in a right-angled triangle. As a resultof fragment analysis, two contradictions arose:• contradiction with the definition of a right triangle
(figure 19);• a contradiction with the theorem on the sum of the
angles of a triangle (figure 20).79
Among the situations that describe the incompletenessof the knowledge base, the following can be distin-guished:• specified the main identifiers of a given entity for
some, but not all external languages;• system identifier is specified for the given entity, but
the main identifiers for all external languages arenot specified;
• the definition or explanation for the concept of thesubject domain is not specified;
• no constants are used in the definition;• the key sc-element of the semantic neighborhood is
not specified;• maximum studied object class is not specified for
subject domain;• relation domains are not specified;• no unit or scale for the measured parameter is
indicated;• concept is not related to any subject domain.To identify incompleteness in the knowledge base,
rules are used that are recorded within the frameworkof the corresponding ontologies in the knowledge base.The figure 21 shows an example of such a rule, accordingto which each relation must have a definition domain.
The listed classes of problem structures are specifiedin the ontology of subject domain of the problem partsof the knowledge base. The specified list of classes canbe expanded and supplemented.
2) Means of specifying changes and transitions inthe knowledge base: In the course of its evolution,the knowledge base undergoes significant changes, inparticular, it is necessary to make changes that affect theconceptual structure of the subject domains described inthe knowledge base. Among these types of changes themost problematic are the following:• in the knowledge base you need to override the
already introduced and used concept.• in the knowledge base an alternative concept ap-
pears, which excludes the use of another conceptassociated with it.
To solve problems in the above situations, the follow-ing classes of concepts are introduced that are part of theontology of situations and events in sc-memory:• main concept;• non-main concept;• concept moving from main to non-main;• concept moving from non-main to main.The figure 22 shows an example of the specification
of the transition process in the knowledge base with theindication of the planned completion dates of this processand the rule on the basis of which the transition is made.
As mentioned earlier, the history of computer systemdevelopment section is used to store the history ofchanges in the knowledge base in the process of its
evolution. Figure 23 shows an example of a structurethat uses means for specifying changes made in theknowledge base.
It should be noted that in this case, actions arespecified not only for editing the knowledge base, butalso actions for coordinating the changes made to it.All performed actions, as well as their specifications,are included into the section history of computer systemdevelopment.
This mechanism for changes fixation in the knowledgebase is the basis for managing versions of the knowledgebase. In the framework of the proposed approach, it isassumed that, if necessary, a rollback of the changesmade before any action in the history is required toperform in reverse order a number of actions, which areinverse to the actions that follow the specified action inthe history. At the same time, actions performed in thisway are also added to the change history in the order ofexecution.
C. Problem solver and user interface of system for thecollective knowledge bases development support
Problem solver of system for the collective knowledgebases development support is a team of knowledgeprocessing agents, each of which automates actions be-longing to any of the classes of actions of the knowledgebases developers discussed above.
The structure of the considered solver in SCn-code:
Problem solver of system for the collective knowledgebases development support<= abstract sc-agent decomposition*:
• Non-atomic sc-agent of knowledge bases editing• Non-atomic sc-agent for automation of activities
of ordinary knowledge base developer• Non-atomic sc-agent for automation of activities
of knowledge base administrator• Non-atomic sc-agent for automation of activities
of knowledge base manager• Non-atomic sc-agent for automation of activities
of knowledge base expert• Non-atomic sc-agent for knowledge base
characteristics calculating
Traditionally, when working collectively with a sharedresource, in this case, a knowledge base, conflicts mayarise, for example, several developers try to enter con-flicting or duplicate information in the knowledge base,try to simultaneously change the same piece of knowl-edge base. The final decision is the responsibility of theknowledge base administrator.
User interface of system for the collective knowledgebases development support is presented by a set ofinterface commands that allow developers to initiate
80
Figure 19. Example of the description of the contradiction with the definition in the knowledge base
Figure 20. Example of the description of the contradiction with the theorem in the knowledge base
81
Figure 21. Example of incompleteness specification in the knowledgebase
the activity of the desired agent that is part of thissystem [77], as well as a set of editors which allowediting knowledge base fragments taking into accountthe mechanism discussed earlier.
In the current version of the tools, there are two editorsthat support the ability to edit in the SCg (figure 24) andSCn (figure 25) languages.
D. Tools of automating the development of problemsolvers sc-models
Among the tasks solved by tools of automating thedevelopment of problem solvers sc-models are technicalsupport for problem solver developers, including ensur-ing the correct and efficient implementation of the stepsprovided by the above methods. These tools are alsoimplemented as an osti-system, which can be used bothin the local version and as a subsystem for the automationsystem for the development of knowledge bases.
In turn, within the framework of the system underconsideration, two subsystems are conventionally dis-tinguished: the subsystem of automation of the processof constructing and modifying of knowledge processingagents and the subsystem of automating the process ofconstructing and modifying of scp-programs.
Graphically, the structure of the system under consid-eration and its subsystems can be represented as follows(figure 26).
An important stage in the development of softwaresystems is the debugging of the developed components.In the case of problem solvers based on OSTIS Technol-ogy, two fundamentally different levels of debugging aredistinguished:• debugging at sc-agents level;
• debugging at scp-program level.In the case of debugging at the sc-agents level, the act
of execution of each agent is considered indivisible andcannot be interrupted. In this case, both atomic sc-agentsand non-atomic ones can be debugged. The initiation ofone or another agent, including one that is not part of anatomic one, is done by creating appropriate constructionsin sc-memory, thus, debugging can be done at differentlevels of detailing of agents, even atomic ones.
Taking into account the fact that the model of agentsinteraction used within the framework of OSTIS Tech-nology uses a universal variant of interaction of agentsthrough common memory, the considered agent designsupport system can serve as a basis for agent modelingsystems that use other communication principles, forexample, direct message exchange between agents.
Debugging at the level of scp-programs is carried outsimilarly to the existing modern approaches to debug-ging procedural programs and suggests the possibility ofsetting breakpoints, step-by-step program execution, etc.
The considered system for automating the constructingand modifying of problem solvers, accordingly, its sc-model, is divided into two more specific ones:
System for automating the constructing and modifyingof problems solvers using OSTIS Technology<= basic decomposition*:
• System for automating the constructing and
modifying of knowledge processing agents• System for automating the constructing and
modifying of scp-programs
In turn, these subsystems are decomposed in accor-dance with the general principles of ostis-systems build-ing into sc-models of the knowledge base, problem solverand user interface. Next, we consider in more detail thecomponents listed.
The knowledge base of the system for automating theconstructing and modifying of problem solvers includes,in addition to the Knowledge base kernel and kernelextensions, sc-models of knowledge bases provided atthe level of OSTIS Technology and models of subjectdomains of scp-programs and scp-interpreter as well thedescription of key concepts related to verification anddebugging of scp-programs such as breakpoint, incor-rectness in the scp-program, error in the scp-programand others.
The problem solver of the system for automating theconstructing and modifying of knowledge processingagents has the following structure:
Problem solver of system for for automating theconstructing and modifying of knowledge processingagents<= decomposition of sc-agent*:
82
Figure 22. Transition process specification in knowledge base
Figure 23. Means of specifying changes made in the knowledge base
83
Figure 24. SCg-editor example
Figure 25. SCn-editor example
• Abstract sc-agent of sc-agents verification<= decomposition of sc-agent*:• Abstract sc-agent of sc-agents specification
verification• Abstract sc-agent for checking a nonatomic
sc-agent for the consistency of itsspecification to the specifications ofparticular sc-agents in its composition
• Abstract sc-agent of sc-agents teams debugging<= decomposition of sc-agent*:• Abstract sc-agent of search for all running
processes corresponding to a given sc-agent• Abstract sc-agent of initiation of a given
sc-agent on the given arguments• Abstract sc-agent of activation of a given
sc-agent• Abstract sc-agent of deactivation of a given
sc-agent• Abstract sc-agent of setting a lock of a
given type for a given process on a givensc-element
• Abstract sc-agent of unlocking of all locksof a given process
• Abstract sc-agent of unlocking of all locksof a given sc-element
In turn, the problem solver of the automation systemfor automating the constructing and modifying of scp-programs has the following structure:
Task solver of the automation system for automatingthe constructing and modifying of scp-programs<= decomposition of sc-agent*:• Abstract sc-agent of scp-programs verification• Abstract sc-agent of scp-programs debugging<= decomposition of sc-agent*:• Abstract sc-agent of launch of a given
scp-program for a given set of input data• Abstract sc-agent of launch of a given
scp-program for a given set of input datain step-by-step mode
• Abstract sc-agent of search of allscp-operators in the scp-program
• Abstract sc-agent of search of allbreakpoints within the scp-process
• Abstract sc-agent of adding breakpoint inscp-program
• Abstract sc-agent of removing breakpoint84
System for automating the constructing and modifying of problem solvers
Subsystem for automating the constructingand modifying of scagents
Subsystem for automating the constructingand modifying of scpprograms
Knowledge
baseInterface Problem
solver
Knowledge
baseInterface Problem
solver
IMS.ostis Metasystem
Library of reusable components of problem solvers
Set of components Means of componentsspecification Tools of components search
Figure 26. The structure of the system for automating the constructing and modifying of problem solvers
from scp-program• Abstract sc-agent of adding breakpoint in
scp-process• Abstract sc-agent of removing breakpoint
from scp-process• Abstract sc-agent to continue the execution
of the scp-process on one step• Abstract sc-agent to continue the execution
of the scp-process to a breakpoint or end• Abstract sc-agent for viewing information
about the scp-process• Abstract sc-agent for viewing information
about the scp-operator
Since the objects of the design of the described
automation system are the components of the problemsolvers, in particular, agents and knowledge processingprograms presented in the SC-code, such a system canuse the basic means of external representation of the textsof the SC-code, for example, SCn or SCg languages.
In order to visually simplify the process of verifyingand debugging the components of the solver, an approachis used that assumes that only the minimum necessary setof sc-elements is displayed to the system user at a time.For example, when debugging a scp-process, it suffices todisplay the scp-operators and connections between them.If necessary, the user can manually request and view thespecification of the desired scp-operator at the time ofthe break. This approach is embedded in the algorithmsof all agents of the described system.
Thus, at present, the user interface of the system forautomating the constructing and modifying of problemsolvers is represented by a set of interface commandsthat allow the user to initiate the activity of the necessary
agent that is part of this system.
X. MEANS OF PROJECT TASK SPECIFICATION
To represent the project tasks and their specificationsin the knowledge base of the system for automating thedevelopment of knowledge bases, the first version of thesc-model of the Subject domain of project actions andits ontology were developed. This subject domains isparticular for the Subject domain of actions and tasks[73] and, thus, inherits from it many general concepts,such as action, action class, task , decomposition ofaction*, subaction*, performer* and others.
In the framework of the Subject domain of projectactions, concepts are studied that are directly related toproject activities, in particular, with classes of projectactivities, priorities of project activities and dependenciesbetween them. The corresponding ontology can becomethe basis for the formalization of existing methods andstandards in the field of organization of project andintelligent activity [78], [79] and was developed takinginto account existing standards in this field.
It should be noted that in the framework of the pro-posed approach to the formalization of activities task istreated as a specification (semantic neighborhood withinthe framework of the knowledge base) of a certainaction. Thus, each action can be assigned a certaintask, containing the conditions in which the specifiedaction is or should be performed. In this connection, itturned out to be inexpedient to introduce separately theclassification of actions and the classification of tasks,and the choice was made in favor of the classificationof actions, since the concept of action can be used ina wider context, which is not necessarily related to theproject activity. Thus, in describing project activities, theconcepts action and action class will be used, and it is
85
understood that, if necessary, a project action can alwaysbe put in correspondence with a project task.
Examples of the specification of concepts studied inthe Subject domain of project actions in the SCn-code:
project action⊂ action
dependent action*∈ binary relation:=> first domain*:
project action=> second domain*:
project action
Tuples of dependent action* relation connect togethersome project action and another project action, whichcannot be completed until the original project action issuccessfully executed. It is assumed that the originalaction is not a subaction for the dependent action.
action priority*∈ binary relation:=> first domain*:
project action=> second domain*:
project action
Tuples of action priority* relation connect two projectactions, the first of which is of higher priority for somereason. Most often it is assumed that both actions are theactions of some general action.
In turn, for the development of ostis-systems, addi-tional classes of project activities were allocated, takinginto account the specifics of developing sc-models ofknowledge bases and sc-models of problem solvers. Thespecified classes of actions are studied in the frameworkof the Subject domain of actions of knowledge bases sc-models developers.
Fragment of the typology of project actions of theostis-systems sc-models developers in the SCn-code:
action. build a new fragment for inclusion in theknowledge base⊃ action. build subject domain sc-model=> abstract subaction*:• action. build a structural specification of
subject domain• action. build a terminological ontology of
subject domain• action. build a set-theoretic ontology of
subject domain• action. build a logical ontology of subject
domain⊃ action. build a semantic neighborhood of a given
entity⊃ action. develop an example of the given concept
use
An example of the specification of project activities forthe development of the IMS.ostis Metasystem knowledgebase:
Section. Development plan of the IMS.ostisMetasystem3 key sc-element’:• Action. develop exercises for the formalization of
basic knowledge• Action. develop a family of introductory sections
on OSTIS Technology• Action. build sc-model of the subject domain of
artificial neural networks=> subaction*• Action. build a structural specification of
the subject domain of artificial neuralnetworks
• Action. build a terminological ontology ofthe subject domain of artificial neuralnetworks
• Action. build a set-theoretic ontology of thesubject domain of artificial neural networks
• Action. build a logical ontology of thesubject domain of artificial neural networks
XI. MEANS OF SPECIFICATION OF PARTICIPANTS INTHE DEVELOPMENT OF SEMANTIC COMPUTER
SYSTEMS
One of the important advantages of the approach to theorganization of the development process, in which allthe information about executing, executed and plannedactions, participants in the process, etc., is recorded in theknowledge base, is the possibility of creating professional”portraits” of developers which can be further analyzedand taken into account when solving, for example, suchtasks as:
• evaluation of the developer’s total contribution tothe development results for a certain period, includ-ing in material terms;
• evaluation of the experience and competence of thedeveloper in solving tasks of certain classes for•• planning available resources and optimizing the
assignment of tasks in terms of their implemen-tation;
•• determining the need to train certain developersin any areas, the improving of certain skills ofa specific developer;
•• assignment of authority and determining thevalue of a particular developer’s opinion whenmaking any collective decisions;
• the ability to automate the above processes.
An example of a fragment of a knowledge base inthe SCg-code describing a specific participant in thedevelopment process (figure 27):
86
Figure 27. An example of the description of a developer professionalportrait
XII. MECHANISMS FOR ASSESSING THECOMPLEXITY OF PROJECT TASKS AND THE
CONTRIBUTION OF THE DEVELOPERS OF SEMANTICCOMPUTER SYSTEMS AND THEIR COMPONENTS
Evaluation of the developer’s experience and the con-struction of his professional portrait are closely relatedto the mechanisms for assessing the contribution ofeach particular developer to the system being developed.At the same time, the assessment of the contribution,expressed in any conventional quantitative units, willallow both to evaluate the developer’s experience anddirectly provide his material remuneration and solvea number of other tasks related to the assessment ofthe participation of developers in the development ofa specific system. Thus, it is important to move fromcontinuous improvement of methods and tools of assess-ing the complexity of project tasks in developing suchsystems, as well as assessing the quality, timeliness andvalue of project developer’s results to improving methodsand tools of project activities stimulating.
In the development of modern computer systems, theassessment of the complexity of the tasks being solved,as a rule, manually on the basis of accumulated experi-ence and is expressed in man-hours. Real remuneration isformed depending on the qualifications of the developerand the total number of man-hours corresponding to thetasks solved for a certain period of time.
When assessing the contribution of the developers ofosti-systems, it is possible to take into account the se-mantics of the fragments being developed, which, on theone hand, will allow automate the process of assessingcontribution for each developer, and on the other, allowto make such an assessment more objective.
In addition, the fact that all project activities aredescribed in the knowledge base of the designed system
and, accordingly, can be analyzed by the system itself,provides additional opportunities for automating the pro-cess of evaluating the contribution of the developer.
The assessment of the contribution in the general casemay depend on the following factors:
• directly the amount of the changes made (thenumber of concepts, the number of sc-arcs, thenumber of operators in programs, etc.) and theamount of work performed (in man-hours or otherconventional units of labor intensity);
• the complexity of the changes (in general, a frag-ment of the knowledge base describing a formallogical statement is considered more complicatedthan, for example, the description of a simple set-theoretic connection between concepts);
• quality of the changes made, which is assessed notonly in terms of the correctness of the changes,but also their completeness, compatibility with otherfragments of the system, etc. As was shown earlier,the assessment of the quality of fragments of knowl-edge bases of ostis-systems (including the specifi-cations of knowledge-processing agents and theircorresponding programs) can be largely automated;
• importance (purposefulness, expediency, priority) ofthe task (accomplishment of the task with a priorityhigher in terms of achieving current goals, is ratedhigher than solving a useful, but not very prioritytask).
One of the problems that arise in assessing the contri-bution of the developer, and in assessing his professionalexperience, is the problem that the author is sometimesfrom the point of view of the system (the person whodirectly formed the proposal to edit the knowledge base)and the real author of knowledge introduced into thesystem (an expert who does not use the technical tools ofthe knowledge base editing) may turn out to be differentpeople, and the formal authorship will be attributed notto the expert, but to his technical assistant. To solvethis problem, in addition to the obvious option, whichinvolves simplifying technical tools and adapting themto subject domain experts, an option is proposed wherethe authorship of the expert is clearly specified manuallyand considered in the same way as any other proposalfor editing the knowledge base.
XIII. CONCLUSION
The paper discusses the principles of developing newgeneration semantic computer systems based on the OpenSemantic Technology for Intelligent System Design (OS-TIS Technology), justifies the advantages of transitionfrom traditional computer systems to semantic computersystems from the point of view of their design process,and considers the advantages of developing design au-tomation tools as semantic computer systems.
87
The main conclusions on the work include the follow-ing:• the accumulated experience of organizing and au-
tomating the development of modern computer sys-tems (synthesis, assembly, analysis, testing, diag-nostics, etc.) is a rich basis for creating models,methods and means of organizing and computersupport for project activities that should be directed:•• on the consistency of project actions of all
developers (compatibility of project results);•• on the reduction of the time of transition from
the current workable version to the next alsoconsistent version not due to the intensificationof the developers’ activities, but due to:• • • increasing the consistency of their ac-
tions;• • • increase the level of valuation, the pur-
posefulness of the results of each de-veloper from the point of view of theearliest construction of the next consis-tent workable version of the developedsystem, which has qualitative advantagesover the previous version;
• the features of the development objects themselves(semantic computer systems) due to the possibilityof their consideration at the semantic level createfavorable prerequisites for effective analysis:•• consistency of project activities;•• quality of project results (consistency, com-
pleteness, clearness);•• valuation (purposefulness) of project results;•• scope of completed design work;This, in turn, creates prerequisites for creatingmethods and tools of effectively stimulating projectactivities, which can also be used in open sourceprojects that assume free entry into the developmentteam;
• development of semantic computer systems in thepresence of a satisfactory version of the imple-mentation of a universal interpreter of semanticmodels is reduced to the development of the rel-evant sections of knowledge base. Therefore, theintelligent system to support the collective designof knowledge bases of semantic computer systemshas a special place in the complex of design toolsfor semantic computer systems.
ACKNOWLEDGMENT
This work was supported by the BRFFR-RFFR (No.F18R-220).
REFERENCES
[1] V. V. Golenkov and N. A. Guljakina, “Proekt otkrytojsemanticheskoj tehnologii komponentnogo proektirovanijaintellektual’nyh sistem. chast’ 1: Principy sozdanija [project of
open semantic technology for component design of intelligentsystems. part 1: Creation principles],” Ontologija proektirovanija[Ontology of design], no. 1, pp. 42–64, 2014.
[2] V. Golenkov, N. Guliakina, N. Grakova, I. Davydenko, V. Niku-lenka, A. Eremeev, and V. Tarassov, “From training intelligentsystems to training their development tools,” in Otkrytye seman-ticheskie tehnologii proektirovanija intellektual’nyh sistem [Opensemantic technologies for intelligent systems], V. Golenkov, Ed.,BSUIR. Minsk , BSUIR, 2018, pp. 81–98.
[3] A. Kolesnikov, Gibridnye intellektual’nye sistemy: Teoriya itekhnologiya razrabotki [Hybrid intelligent systems: theory andtechnology of development], A. M. Yashin, Ed. SPb.: Izd-voSPbGTU, 2001.
[4] I. Davydenko, “Semantic models, method and tools of knowledgebases coordinated development based on reusable components,”in Otkrytye semanticheskie tehnologii proektirovanija intellek-tual’nyh sistem [Open semantic technologies for intelligent sys-tems], V. Golenkov, Ed., BSUIR. Minsk , BSUIR, 2018, pp.99–118.
[5] D. Shunkevich, “Agent-oriented models, method and tools ofcompatible problem solvers development for intelligent systems,”in Otkrytye semanticheskie tehnologii proektirovanija intellek-tual’nyh sistem [Open semantic technologies for intelligent sys-tems], V. Golenkov, Ed., BSUIR. Minsk , BSUIR, 2018, pp.119–132.
[6] (2018, Feb.) Autodesk generative design. [Online]. Available:https://www.autodesk.com/solutions/generative-design
[7] (2018) Autodesk dreamcatcher. [Online]. Available: https://autodeskresearch.com/projects/dreamcatcher
[8] S. M. Duffy and A. H. B. Duffy, “Sharing the learning activ-ity using intelligent cad,” Artificial Intelligence for EngineeringDesign, Analysis and Manufacturing, vol. 10, no. 2, p. 83–100,1996.
[9] L. S. Cardona-Meza and G. Olivar-Tost, “Modeling and simula-tion of project management through the PMBOK standard usingcomplex networks,” Complexity, vol. 2017, pp. 1–12, 2017.
[10] J. Stark, Product Lifecycle Management (Volume 1): 21st CenturyParadigm for Product Realisation, 3rd ed. Switzerland: Springer,Cham, 2015.
[11] T. Dillon, E. Chang, and P. Wongthongtham, “Ontology-basedsoftware engineering- software engineering 2.0.” Australian Soft-ware Engineering Conference, IEEE Computer Society, pp. 13–23, 2008.
[12] A. Emdad, “Use of ontologies in software engineering,” 2008,pp. 145–150.
[13] A. N. Andrichenko, “Tendencii i sostojanie v oblasti upravlenijaspravochnymi dannymi v mashinostroenii [trends and status in thefield of reference data management in mechanical engineering],”Ontologija proektirovanija [Ontology of design], no. 2(4), pp.25–35, 2012.
[14] A. Fedotova, I. Davydenko, and A. Pfortner, “Design intelligentlifecycle management systems based on applying of semantictechnologies,” vol. 1. Switzerland , Springer InternationalPublishing„ 2016, pp. 251–260.
[15] (2018) Agile manifest. [Online]. Available: http://agilemanifesto.org/iso/ru/manifesto.html
[16] P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta, “Agilesoftware development methods: Review and analysis,” AgileSoftware Development, vol. 1, pp. 31–59, 2002.
[17] (2018) Best version control systems. [Online]. Available:https://www.g2crowd.com/categories/version-control-systems
[18] (2018) Comparison of issue-tracking systems. [On-line]. Available: https://en.wikipedia.org/wiki/Comparison\_of\_issue-tracking\_systems
[19] (2018) Project management software. [Online]. Available:https://www.capterra.com/project-management-software
[20] (2018) Comparison of gui testing tools. [Online]. Available: https://en.wikipedia.org/wiki/Comparison\_of\_GUI\_testing\_tools
[21] (2018) List of web testing tools. [Online]. Available: https://en.wikipedia.org/wiki/List\_of\_web\_testing\_tools
[22] (2018) Best version control hosting software. [Online]. Available:https://www.g2crowd.com/categories/version-control-hosting
88
[23] R. Iqbal, A. Murad, and A. Mustapha, “An analysis of ontologyengineering methodologies : a lit. rev.” Research J. of Appl.Sciences, Engineering a. Technology, vol. 6, no. 16, pp. 48–62,2013.
[24] P. Sainter, K. Oldham, and A. Larkin, “Achieving ben-efits from knowledge-based engineering systems in thelonger term as well as in the short term,” SemanticScholar, retrieved from https://pdfs.semanticscholar.org/0edc/b90ca6601192e10d7eb2fd03147fb72e1f9d.pdf.
[25] A. A. Slobodjuk, S. I. Matorin, and S. N. Chetverikov, “Opodhode k sozdaniju ontologij na osnove sistemnoobektnyhmodelej predmetnoj oblasti [on the approach to the creationof ontologies based on system-object domain models],” Nauch.vedomosti Belgor. gos. un-ta. Ser.: Jekonomika. Informatika[Scientific journ. of Bel. state univ. Ser .: Economy. Computerscience], no. 22, pp. 186–193, 2013.
[26] M. Uschold and M. Grueninger, “Ontologies: principles, methodsand applications,” The Knowledge Engineering Rev, vol. 11, no. 2,pp. 93–136, 1996.
[27] M. Gruninger and M. Fox, “Methodology for the design andevaluation of ontologies [Electronic resource],” Workshop onBasic Ontological Issues in Knowledge Sharing, IJCAI-95,mode of access: http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=70152DBAFA34B86E22661C3E3F7C978F?doi=10.1.1.44.8723\&rep=rep1\&type=pdf. — Date of access: 30.10.2016.
[28] A. Gomez-Perez, M. Fernandez-Lopez, and A. de Vicente, “To-wards a method to conceptualize domain ontologies,” in ECAI96 : proc. of the 12th Europ. Conf. on Artificial Intelligence,Budapest, 11–16 Aug. 1996, Europ. Coordinating Comm. forArtificial Intelligence ; ed. W. Wahlster. Budapest, 1996, pp.41–51.
[29] A. Gomez-Perez and M. Suarez-Figueroa, “Scenarios for buildingontology networks within the neon methodology,” in Proceedingsof the Fifth International Conference on Knowledge Capture (K-CAP 2009), Assoc. for Computing Machinery, Spec. InterestGroup on Artificial Intelligence. New York, 2009, pp. 183–184.
[30] Y. Sure, M. Erdmann, J. Angele, S. Staab, R. Studer, andD. Wenke, “Ontoedit: collaborative ontology development for thesemantic web,” in The Semantic Web (ISWC 2002) : proc. ofthe first Intern. semantic web conf., Sardinia, 9–12 June 2002,I. Horrocks and J. Hendler, Eds. Berlin, 2002, pp. 221–235.
[31] A. Bernaras, I. Laresgoiti, and J. Corera, “Building and reusingontologies for electrical network applications,” in ECAI 96 : proc.of 12th Europ. Conf. on Artificial Intelligence, Budapest, 11–16Aug. 1996, Europ. Coordinating Comm. for Artificial Intelligence.Chichester, 1996, pp. 298–302.
[32] H. S. Pinto, S. Staab, and C. Tempich, “Diligent: Towards afine-grained methodology for distributed, loosely-controlled andevolving engineering of ontologies,” in European Conference onArtificial Intelligence (ECAI’04) : proc. of the 16th Conference,Valencia, Spain – August 22–27, 2004. Amsterdam, 2004, pp.393–397.
[33] B. Swartout, R. Patil, K. Knight, and T. Russ, “Toward distributeduse of large-scale ontologies [Electronic resource],” SemanticScholar, mode of access: https://www.semanticscholar.org/paper/Toward-Distributed-Use-of-Large-Scale-Ontologies-Swartout-Patil/07b64a07e6d56a91f2471975a3922e3fcd9ff2d7?tab=abstract. —Date of access: 26.08.2016.
[34] A. D. Nicola, M. Missikoff, and R. Navigli, “A proposal fora unified process for ontology building: Upon,” in Databaseand expert systems applications : proc. of the 16th intern. conf.(DEXA 2005), Copenhagen, 22–26 Aug. 2005, K. Andersen,J. Debenham, and R. Wagner, Eds. New York ; Berlin , Springer,1997, pp. 655–664.
[35] J. V. Rogushina, “Semanticheskie wiki-resursy i ih ispol’zovaniedlja postroenija personificirovannyh ontologij [semantic wiki-resources and their use for building personalized ontologies],” inProceedings of the 10th International conference of programming(UkrPROG’2016), Kyiv, 24–25 May 2016, P. A. I. Sergienko, Ed.Kyiv, 2016, pp. 188–195.
[36] M. Raman, “Wiki technology as a «free» collaborative tool withinan organizational setting,” Inform. Systems Management, vol. 23,no. 4, pp. 59–66, 2006.
[37] “MediaWiki [Electronic resource],” mode of access: https://www.mediawiki.org.. — Date of access: 26.11.2016.
[38] A. J. Gladun and J. V. Rogushina, “Wiki-tehnologii,” Telekom.Kommunikacii i seti, no. 5, p. 58, 2008.
[39] Semantic MediaWiki [Electronic resource]. Mode of ac-cess: https://www.semantic-mediawiki.org. — Date of access:21.10.2016.
[40] I. S. Chistjakova, “Inzhenerija ontologij [ontology engineering],”Inzhenerija programm. zabezpechennja [Software engineering],no. 20, pp. 51–66, 2014.
[41] “Protege [Electronic resource],” mode of access: http://protege.stanford.edu. — Date of access: 27.05.2016.
[42] J. Euzenat, “Corporate memory through cooperative creation ofknowledge bases and hyper-documents [Electronic resource],”Knowledge Acquisition for Knowledge-Based Systems : Proc.of 10th Workshop, EKAW’97, mode of access: http://ksi.cpsc.ucalgary.ca/KAW/KAW96/euzenat/euzenat96b.html. — Date ofaccess: 30.10.2016.
[43] “Ontolingua [Electronic resource],” Knowledge systems, al lab-oratory, mode of access: http://www.ksl.stanford.edu/software/ontolingua. — Date of access: 29.05.2016.
[44] “Building DAML+OIL ontologies [Electronic resource],” OilEd,mode of access: http://oiled.semanticweb.org/building. — Date ofaccess: 24.05.2016.
[45] “Projects [Electronic resource],” Knowledge Media Institute,mode of access: http://kmi.open.ac.uk/projects. — Date of access:22.05.2016.
[46] N. M. Borgest, “Rol’ ontologii v proektirovanii informacionnyhsistem [role of ontology in information system design],” in Otkry-tye semanticheskie tehnologii proektirovanija intellektual’nyhsistem [Open semantic technologies for intelligent systems],V. Golenkov, Ed., BSUIR. Minsk , BSUIR, 2014, pp. 155–160.
[47] O. M. Ovdej and G. J. Proskudina, “Obzor instrumentovinzhenerii ontologij [ontology engineering tools survey],”Jelektron. b-ki. [Electronic libraries], vol. 7, no. 4, 2004, availableat: http://www.elbib.ru/index.phtml?page=elbib/rus/journal/2004/part4/op. — accessed: 30.11.2016.
[48] E. Alatrish, “Comparison some of ontology,” Management In-form. Systems, vol. 8, no. 2, pp. 18–24, 2013.
[49] N. Noy, M. Klein, S. Kunnatur, A. Chugh, and S. Falconer,“PROMPT [Electronic resource],” Protege Wiki, mode of ac-cess: https://protegewiki.stanford.edu/wiki/PROMPT. — Date ofaccess: 23.11.2016.
[50] “Chimaera [Electronic resource],” Knowledge systems, al lab-oratory, mode of access: http://www.ksl.stanford.edu/software/chimaera/. — Date of access: 20.05.2016.
[51] “OntoMerge: ontology translation by merging ontologies[Electronic resource],” Computer Science, mode of access: http://cs-www.cs.yale.edu/homes/dvm/daml/ontology-translation.html.— Date of access: 19.10.2016.
[52] H. Chalupsky, “OntoMorph: a translation system for sym-bolic knowledge [Electronic resource],” mode of access: https://www.isi.edu/~hans/ontomorph/presentation/ontomorph.html. —Date of access: 19.05.2016.
[53] “Ontology based system enhanced with relationships for vocabu-lary heterogeneity resolution [Electronic resource],” SID ResearchGroup, mode of access: http://sid.cps.unizar.es/OBSERVER. —Date of access: 16.05.2016.
[54] G. Stumme and A. Maedche, “Fca-merge: bottom-up merging ofontologies,” in IJCAI-01 : proc. of the seventeenth Intern. jointconf. on artificial intelligence, Seattle, 4–10 Aug. 2001, B. Nebel,Ed., Amer. Assoc. for Artificial Intelligence. San Francisco,2001, pp. 225–230.
[55] M. Vargas-Vera, E. Motta, J. Domingue, M. Lanzoni, A. Stutt,and F. Ciravegna, “MnM: ontology driven semi-automatic andautomatic support for semantic markup,” in Knowledge engineer-ing and knowledge management: ontologies and the semanticweb : proc. of the 13th intern. conf., Siguenza, 1–4 Oct. 2002,A. Gomez-Perez and V. R. Benjaminsns, Eds. Berlin ; NewYork, 2002, pp. 379–391.
[56] J. Heflin and J. Hendler, “A portrait of the semantic web inaction,” IEEE Intelligent Systems, vol. 16, no. 2, pp. 54–59, 042001.
89
[57] I. V. Efimenko and V. F. Horoshevskij, Ontologicheskoemodelirovanie jekonomiki predprijatij i otraslej sovremennojRossii : v 2 t. M. : Nac. issled. un-t «Vyssh. shk. jekonomiki»,2011, vol. 1 : Ontologicheskoe modelirovanie: podkhody, modeli,metody, sredstva, resheniya [Ontological modeling: approaches,models, methods, tools, solutions], (Preprint / Nac. issled. un-t «Vyssh. shk. jekonomiki» ; Serija: Matematicheskie metodyanaliza reshenij v jekonomike, biznese i politike ; WP7/2011/08).
[58] A. F. Tuzovskij, I. A. Zaikin, and V. Z. Jampol’skij, “Sistemakollektivnoj podderzhki ontologicheskih modelej [system ofcollective support of ontological models],” Izv. Tom. politehn. un-ta [News of Tomsk polytechnic univ.], no. 5, pp. 89–94, 2011.
[59] T. Tudorache, N. Noy, S. Tu, and M. Musen, “Supportingcollaborative ontology development in protege,” in The semanticweb (ISWC 2008) : proc. of the 7th Intern. semantic web conf.,Karlsruhe, 26–30 Oct. 2008, D. H. [et al.], Ed. Karlsruhe, 2008,pp. 17–32.
[60] V. V. Gribova, A. S. Kleshchev, F. M. Moskalenko, V. A. Tim-chenko, L. Fedorishchev, and E. A. Shalfeeva, “Platforma dlyarazrabotki oblachnykh intellektual’nykh servisov [platform forintelligent cloud services development],” in XV Natsional’nayakonferentsiya po iskusstvennomu intellektu s mezhdunarodnymuchastiem (KII-2016), Smolensk, 3–7 oktyabrya 2016 g. : trudy: v 3 t. [XV National Conference on Artificial Intelligence withInternational Participation (CAI-2016), Smolensk, October 3–7,2016: procs: 3 v.], vol. 1, Ros. assots. iskusstv. intellekta, Feder.issled. tsentr «Informatika i upravlenie» Ros. akad. nauk [Russ.assoc. art. intelligence, Feder. research Center «Informatics andManagement» Rus. Acad. of science]. Smolensk, 2016, pp. 24–33.
[61] A. N. Borisov, “Postroenie intellektual’nyh sistem, osnovannyh naznanijah, s povtornym ispol’zovaniem komponentov [building ofintelligent knowledge-based systems with reusable components],”in Otkrytye semanticheskie tehnologii proektirovanijaintellektual’nyh sistem [Open semantic technologies forintelligent systems], V. Golenkov, Ed., BSUIR. Minsk : BSUIR,2014, pp. 97–102.
[62] L. S. Globa and R. L. Novogrudskaja, “Modeli i metodyintegracii informacionnyh i vychislitel’nyh resursov [modelsand methods for the integration of information and computingresources],” in Otkrytye semanticheskie tehnologii proektirovanijaintellektual’nyh sistem [Open semantic technologies forintelligent systems], V. Golenkov, Ed., BSUIR. Minsk , BSUIR,2012, pp. 447–452.
[63] G. B. Zagorul’ko and J. A. Zagorul’ko, “Podhod k organizaciikompleksnoj podderzhki processa razrabotki intellektual’nyhsppr v slaboformalizovannyh predmetnyh oblastjah [approachto the organization of complex support of intelligent dmsdevelopment in weakly-structured subject domains],” in Otkrytyesemanticheskie tehnologii proektirovanija intellektual’nyhsistem [Open semantic technologies for intelligent systems],V. Golenkov, Ed., BSUIR. Minsk , BSUIR, 2016, pp. 61–64.
[64] L. S. Bolotova, Sistemy iskusstvennogo intellekta: modelii tehnologii, osnovannye na znanijah : uchebnik [ArtificialIntelligence Systems: Knowledge-Based Models andTechnologies: Textbook]. Moskow , Finances and Statistics,2012.
[65] (2016, Aug.) RDF 1.1 semantics. [Online]. Available: https://www.w3.org/TR/2014/REC-rdf11-mt-20140225
[66] (2016, May) Ontaria: easy access to the semantic web. [Online].Available: https://www.w3.org/2004/ontaria/
[67] (2016, Aug.) Schema.org. [Online]. Available: http://schema.org[68] M. D’Aquina and N. F. Noyb, “Where to publish and
find ontologies? A survey of ontology libraries [Elec-tronic resource],” Web Semantics, vol. 11, 2012, mode ofaccess: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3293483/pdf/nihms-324445.pdf. — Date of access: 20.05.2016.
[69] C. Debruyne, P. D. Leenheer, and R. Meersman, “A method andtool for fact type reuse in the dogma ontology framework,” in Onthe move to meaningful Internet systems: OTM 2009 : in 2 pt.,ser. 2 pt. London ; Berlin , Springer, 2009, pp. 1147–1164.
[70] P. D. Leenheer and C. Debruyne, “Dogma-mess: A tool for fact-oriented collaborative ontology evolution,” in On the move to
meaningful Internet systems: OTM 2008 Workshops, R. Meers-man, Z. Tari, and P. Herrero, Eds. Heidelberg , Springer, 2009,pp. 797–806.
[71] M. Bergman, “Knowledge-based artificial intelligence[Electronic resource],” AI3: Adaptive Information,mode of access: http://www.mkbergman.com/1816/knowledge-based-artificial-intelligence. — Date of access:14.10.2016.
[72] A. Filippov, V. Moshkin, D. Shalaev, and N. Yarushkina,“Edinaya ontologicheskaya platforma intellektual’nogo analizadannykh [unified ontological data mining platform],” in Otkrytyesemanticheskie tehnologii proektirovanija intellektual’nyhsistem [Open semantic technologies for intelligent systems],V. Golenkov, Ed., BSUIR. Minsk , BSUIR, 2016, pp. 77–82.
[73] (2018, Nov.) IMS metasystem. [Online]. Available: http://ims.ostis.net/
[74] V. Golenkov, “Ontology-based design of intelligent systems,”in Otkrytye semanticheskie tehnologii proektirovanija intellek-tual’nyh sistem [Open semantic technologies for intelligent sys-tems], V. Golenkov, Ed., BSUIR. Minsk , BSUIR, 2017, pp.37–56.
[75] V. P. Ivashenko, “Modeli i algoritmy integratsii znanii na osnoveodnorodnykh semanticheskikh setei [models and algorithms forthe knowledge integration based on homogeneous semanticnetworks],” Otkrytye semanticheskie tekhnologii proektirovaniyaintellektual’nykh system [Open semantic technologies forintelligent systems], vol. 2, pp. 111–132, 2015.
[76] (2018, Nov.) Github. [Online]. Available: https://github.com[77] D. Koronchik, “Semanticheskie modeli mul’timodal’nykh
pol’zovatel’skikh interfeisov i semanticheskaya tekhnologiyaikh proektirovaniya [semantic models of multimodal userinterfaces and semantic technology for their design],” in Otkrytyesemanticheskie tehnologii proektirovanija intellektual’nyhsistem [Open semantic technologies for intelligent systems],V. Golenkov, Ed., BSUIR. Minsk , BSUIR, 2012, pp. 339–346.
[78] (2018) Pmbok R© guide and standards. [Online]. Available:https://www.pmi.org/pmbok-guide-standards
[79] A. S. Kleshhjov and E. A. Shalfeeva, “Ontologija zadach in-tellektual’noj dejatel’nosti [ontology of the tasks of intelligentactivity],” Ontologija proektirovanija [Ontology of design], vol. 5,no. 2(16), pp. 179–205, 2015.
ПРИНЦИПЫ ОРГАНИЗАЦИИ ИАВТОМАТИЗАЦИИ ПРОЦЕССА РАЗРАБОТКИ
СЕМАНТИЧЕСКИХ КОМПЬЮТЕРНЫХСИСТЕМ
Голенков В.В., Шункевич Д.В.,Давыденко И.Т., Гракова Н.В.
Работа посвящена принципам разработки семанти-ческих компьютерных систем нового поколения на ос-нове Открытой семантической технологии проектиро-вания интеллектуальных систем (Технологии OSTIS).Обоснованы преимущества перехода от традиционныхкомпьютерных систем к семантическим компьютер-ным системам с точки зрения процесса их проектиро-вания, а также рассмотрены преимущества реализациисредств автоматизации проектной деятельности каксемантических компьютерных систем.
Received 09.01.19
90
Principles of decision-making systems buildingbased on the integration of neural networks and
semantic modelsVladimir Golovko
Alexsander KroshchankaBrest State Technical University
Brest, [email protected]
Valerian IvashenkoMikhail Kovalev
Belarusian State University of Informaticsand Radioelectronics
Minsk, [email protected]
Valery TaberkoDzmitry Ivaniuk
JSC Savushkin productBrest, Belarus
Abstract—This article reviews the benefits of integrationneural network and semantic models for building decision-making systems. There purposed an approach to theintegration of artificial neural networks with knowledgebases by inputs and outputs and the specification of thesenetworks in the knowledge base using the ontology ofthe respective subject domains. The proposed approachis considered on the real production problem of JSCSavushkin Product for quality control of marking onproducts.
Keywords—ANN, knowledge base, integration, inference,decision-making
INTRODUCTION
Despite the significant results obtained in differentdirections of research in the field of artificial intelligence,the problem of integration of such results is gainingmore and more relevance. A large number of problemsthat should be solved by modern intelligent systems (IS)require the joint use of different models of problemsolving and models of knowledge representation. In turn,the integration of different models of this kind within asingle system often presents significant challenges dueto the development’s isolation of these models.
Currently, one of the most actively developed direc-tions in the field of artificial intelligence is the directionrelated to problem solving based on machine learningmethods. The popularity of methods for problem solv-ing based on machine learning is largely due to thedevelopment of theoretical models of artificial neuralnetworks (ANNs) and productive hardware platforms fortheir implementation. The variety of architectures, meth-ods, directions and ways of ANNs using is constantlyincreasing.
However, it should be noted that not all problems areconvenient to solve with the using of machine learningbecause the complexity of modern tasks creates the needto integrate different approaches to problem solving. It
is the common situation when systems, that use neuralnetwork algorithms, are in need for additional semanticanalysis of results of ANNs work and decision-makingon the basis of this analysis.
In this regard, there is a need to develop approaches tothe building of systems that can use both neural networkand semantic models, as well as able to combine thesemodels in the search for problem solving. There are twomain requirements for such a system:
• flexibility in adding new models;• adaptivity to existing models changing.This article will consider an approach to the building
of such systems on the example of a subsystem ofmarking quality control for the company JSC ”Savushkinproduct”.
I. FORMULATION OF THE PROBLEM
A. General statement of the problem
This article discusses the use of integration of artificialneural networks with knowledge bases to solve problemsof a particular class, the general condition of which canbe formulated as follows: it is necessary to perform asemantic analysis of the results of the machine visionsystem with use of the knowledge available in the system.
In the formal way the general condition can be for-mulated as shown below. There is a signal of fixed sizes. For the machine vision system it is required to findthe transformation f of this signal satisfying the set ofconstraints Rf in the feature space [1]:F q∗d
F ⊆ S × Vwhere q is the number of simultaneously tested signals,
d is the length of the time interval, S is the set of signals,F is the set of product features, V is the set of featurevalues. Moreover, it is required that f should be suchthat there is a transformation g:
91
g ⊆ V q∗d ×D,satisfying a set of constraints Rg , where D is a set of
decisions, the simplest of which are:• notifying the operator;• stopping the process;• making fixes in some devices if possible;• moving the video camera;• cleaning the video camera;
B. Approach to solving the problem
This problem solution consists of two parts, describedby transformations f and g, the implementation of whichmust be integrated to obtain a system that provides acomplete decision. In this paper, it is proposed to usemodels based on artificial neural networks to implementthe transformation f , respectively, to implement thetransformation g, it is proposed to use ontologies baseddecision-making models. Thus, the results obtained bythe authors in [1], devoted to the integration of ANNswith knowledge bases, will be used in this work.
This integration can be considered at different stagesof building partial decisions:
• the training of ANNs for recognition;• data supply to the input of ANNs;• ANNs work results processing• decision development on action or inaction on the
basis of the knowledge base.The first step is to select and train the model to solve
the first part of the problem. Integration is appropriateif these choices and training are partially automated. Forexample, under the condition that there is a system withthe knowledge base, which knows what models (ANNs)can be used to solve the formulated problem and compareon the grounds that there are restrictions in Rf . Or, thesystem knows which training methods can be applied andfor which data sets (training samples), in which case thesystem can manage the training process by passing thisdata for training and testing.
The second stage is also reduced to ANNs manage-ment and integration is appropriate in the presence ofa system with the knowledge base, storing knowledgeabout the input data (eg, video signal).
At the stage of processing the results of ANNs, ifthe network solves the problem approximately, and notexactly, there may be situations when the results arenot validated on the data that were not included inthe training (or test) sample. In this case, integrationis possible if the knowledge base contains knowledgeabout the required feature properties from the set ofRf constraints. If invalid conditions are detected, thenetwork can be adapted.
At the last stage, the feasibility and complexity ofintegration depend on the type of model used for thesolution. In this case, the options for selecting the modelare as follows:
• ANNs similar to ANNs of the first subtask;• ANNs is other than ANNs of the first subtask;• not adaptable (trainable) model that are not related
to ANNs;• adaptable (trainable) model that is not related to
(being) ANNs.The first option does not require serious integration or
requires integration by inputs and outputs, the secondoption requires integration by inputs and outputs, thethird and fourth options require the most serious workon the integration of models. The fourth option, unlikethe third one, does require the use of flexible knowledge-based systems with developed means of knowledge rep-resentation and adaptation of the knowledge base. Thecomplexity of integration in this case will depend on thechoice of a specific model to solve the second sub-task(decision-making). These models can be attributed to:
• problem solving strategies as decision trees andothers;
• classical and non-classical inference models, includ-ing fuzzy and Bayesian models of reasoning, non-monotonic reasoning systems etc.;
• and others.
II. EXISTING APPROACHES TO SOLVING THEPROBLEM
A. Integration approaches
The main approaches to integration are the following:• integration by inputs and outputs, including control
and messaging transfer;• integration by immersion, embedding one model
into another when one model is modeled (inter-preted) by another.
Details of the implementation of these approachesdepend on the complexity of selected integrated models:language, set of states and set of operations. For complexmodels, immersion is a labor-intensive process, so thechoice of integration by inputs and outputs can be dueto reduced labor costs.
The advantages of the approach of integration byinputs and outputs are the possibility of reducing thelabor costs of integration and higher performance of thedecision making.
B. Problems encountered in the task solving
The main problems that are solved by the modelsintegration:
• the reduction of labor costs for the complete deci-sion making model development;
• satisfying restrictions on transformations describingdecisions, including its computational complexity,in order to improve the efficiency and performanceof the entire system;
• obtaining a decision that has a model, i.e. thedecision implementation is not just a computational
92
process, but a formula that has a model semanticsin a certain model of knowledge representation withthe necessary properties, in order to conduct a se-mantic analysis of the obtained decision and explainits results and continue the process of adaptation andintegration.
The benefits are reduced labor costs, finding the mostefficient (productive) solution, and building a flexiblesystem focused on these benefits.
III. PROPOSED APPROACH
A. Detailed problem descriptionThe task of marking control is described in detail in
[1]. Brief description – the marking control module isinstalled on the bottling line. It consists of a cameraconnected to an industrial computer. The camera is fixedin the box and installed above marked covers. The systemshall ensure continuous monitoring of marking quality.Next, we consider in more detail the analysis of the mainreasons for obtaining defective marking and typical waysto eliminate them.
• no ink. If empty bottles start to go, it means theprinter is out of paint. The system can access itadditionally (since the printer is connected to thenetwork) and check the availability of ink.
• camera shift. the system knows that batch bottlinghas started, but there are no positive recognitionresults from the camera.
• incorrect marking. The marking is recognized,transferred to the system, but it does not match thereference – so there was an error when specifyingthe text marking and it is necessary to stop thebottling process and notify the operator.
• unreadable marking. The marking is not recog-nized – one or more digits are not recognized, sothe printer nozzles are clogged – you need to stopthe filling process and notify the Instrumentation en-gineer about the need to clean the printer nozzles. Itis desirable to remove the bottles with the incorrectmarking of the pipeline.
B. TechnologyThe proposed approach is based on the OSTIS tech-
nology ( [2], [3], [4]) and its principles. The OSTIStechnology uses models of knowledge representation andprocessing focused on the unification and work withknowledge at the semantic level. The basic principlesand models used in the approach include:
• knowledge integration model and unified semanticknowledge representation model [1], which is basedon the SC-code [2];
• principles of situational management theory [5];• ontological model of events and facts in knowledge
processing [6];• multi-agent approach [7];• hybrid knowledge processing models [8].
IV. ARCHITECTURE OF THE PROPOSED SYSTEM
As mentioned above, the proposed subsystem of mark-ing quality control is developed on OSTIS technology.
The system, developed on OSTIS technology, is calledostis-system. Each ostis-system consists of a platform-independent implementation of a unified logical-semanticmodel of the system (sc-model of a computer system)and a platform for the interpretation of such models.In turn, each sc-model of a computer system can bedecomposed into sc-model of knowledge base, sc-modelof problem solver, sc-model of interface and abstract sc-memory which stores the constructions of the SC-code[2].
Based on this, the developed system, like any ostis-system, has three main parts:
• sc-model of interface. Describes the SCADA-system project [9], which can be used to track thedecisions made by the system, including decisionsabout the need of an engineer’s involving. Alsohere the engineer can set a sample to configure thesystem to recognize markings on new products.
• sc-model of knowledge base. Describes the knowl-edge base [10] of the system, which contains all thenecessary knowledge for decision making, such aslogical rules, statements, current markings, devicestates and etc. The knowledge base of the systemof quality control marking is described in the VIsection.
• sc-model of problem solver. Describes problemsolver based on the multi-agent approach ( [8],[11]). Contains a set of internal and external agentsthat work with the knowledge base. With the help ofthese agents, problem solver can initiate the recog-nition process, implement the reverse inference ([12], [13]) on the recognition results, call externalprograms for decisions implementation, prepare adecision for the engineer’s terminal. The problemsolver of the subsystem of quality control markingis considered in more detail in the VII section.
Also system has additional modules such as:
• device serving image marking. For the knowledgebase, these devices are represented by agents thatare initiated by the recognition request.
• marking recognition module. The task of this mod-ule is to localize and recognize product markings onthe image. This module is described in more detailin V section.
• robotic subsystem control module. This module hasaccess to the subsystem that directly carries outthe marking of products. The task of this moduleis implementation of system decisions that can betaken without the involvement of the engineer, suchas marking updates, switching the subsystem on andoff, etc.
93
The figure 1 shows the general diagram of all modulesinteraction of the system of marking quality control.
subsystem of marking quality control
Module of computervision
Decisionmaking systemTerminal
engineerKnowledge base
Image
Requests aboutthe current stateof the system,control actions
Problem reports,responses to requests
artificial neuralnetwork
Strategy of decisionmaking (reverseinference)Interpretation of logical rulesAssociative search
Markingrecognitionresults
Module of roboticinstallation control
Requests ofprogram
interpretation
Informationabout faults Interpretation of
managementprograms
Figure 1. Diagram of all modules interaction of the system of qualitycontrol marking
V. STRUCTURE OF THE RECOGNITION MODULE
To solve the problem of caps marking detection we usetwo different approaches. One of them is a componentof the real system, which already used in the work.Another approach is still being explored and this sectionis dedicated to it.
To solve caps detection task we used deep neuralnetwork SSD (Single-shot detector) [14]. A main featureof this architecture is that it detects objects in images inone pass (one-look), without solving two independenttasks (localization and classification). Using of suchnetwork gets acceptable speed of objects detection, andwith modified classificator (for example, Tiny SSD orMobileNet SSD) allows to work in real-time mode ( [15],[16]). Use of networks such as Faster-RCNN [17] givesbetter efficiency, but fundamentally unacceptable for real-time applications because of the high resource intensity[18].
SSD-network as the YOLO-network [19], belongs tocategory of one-look methods, which solve detection taskwith using only one network. A schematic representationof the architecture is shown in figure 2. SSD has a typicalstructure inherent to convolutional neural networks [20].
Figure 2. SSD-network architecture
On output of SSD-network we gets coordinates ofrectangle boxes, which contains objects and labels foreach box, which represent class of object.
We note main features of this network.• It differs from other single shot detectors (in par-
ticular, from YOLO) in that each layer of the
model participates in the formation of informationabout objects and their location (taking into accountthe scale of these objects – each subsequent layerdetects objects of larger size than the previous one)(figure 3);
• It uses pretrained neural network (VGG16 [21],ResNet50 [22], etc.) as a base item, which is in-tegrated into the SSD network and forms featuresmaps that are used to make decisions about theposition and class of objects. These networks trainedon massive set of images. As a alternative path to getpretrained network is a use of special pretrain pro-cedure [23]. Layers for classification are discarded;
• The network uses the Non-Maximum Suppressionalgorithm to reduce the number of generated boxes;
• Each item of feature map forms set of default boxes(or anchors), which differ in scale and aspect ratio.
• Network is trained until for each anchor gets rightprediction for its class and offset.
Although SSD-network with VGG16 works faster incontrast with networks, which use a two-stage process,this architecture cannot be considered as a workingversion to detect in a simple mobile systems. In thiscase it seems appropriate to use the MobileNet as a basicconvolution network. MobileNet can significantly speedup SSD-network, therefore makes possible to process thevideo stream received from the camera over the pipelinein real-time mode. The main feature, due to which anincrease in processing speed is achieved, is that Mo-bileNet contains fewer number of parameters comparedto VGG16, but it is not inferior to it in efficiency [16].Reducing the number of parameters in turn is achievedby using a Depthwise Separable Convolution.
Figure 3. Localization of objects on feature maps of different sizes[14]
Structure of the caps detection system. SSD-networkhas one significant drawback – poor ability to detectsmall objects in the image ( [24], [25]) This is dueto the fact that the feature maps of this network havelow resolution. Therefore, when designing a system forthe detection of caps and labels with the subsequentrecognition of the individual chars, which are includedin the label, we must perform the decomposition of thegeneral task into two subtasks.
• Detection of caps and labels. At this stage the94
missing labels are detected and a decision is madeto send message to the operator about incorrect cap.
• Detection and recognition separate chars. The de-tection and recognition of chars and the formationof the marking representation are perform at thisstage.
Both of these tasks can be solved using SSD-network.But in the case of using one detector for detectionand recognition of all objects of interest (caps, labels,individual characters) is impossible to ensure acceptablequality of work for individual characters, because theoriginal image will be scaled to the size, which makesthe detection of the symbols a difficult task.
Therefore we used two separate SSD-networks fordetection of caps/labels and recognition chars. First net-work detects two classes of objects (caps and labels),then feeds image of label to second network in originalquality.
Description of training set and features of labeling Tosolve the task of detection of caps and labels, the trainingset preparation consisted in manual processing of imageswith the definition for each image characteristics ofrectangular areas (boxes), which include caps and labels(length, width, coordinates of the upper left corner).
As a result, a total set of 940 images was prepared,80% of which form a training set, and the remaining 20%- a test set.
Example of image labeling is presented in fig. 4.
Figure 4. Example of image labeling
Estimation of recognition efficiency The mAP (meanaverage precision metric) was used to evaluate the de-tection quality of caps and labels. This metric is the defacto standard of metrics used to evaluate the qualityof models used for detection [26]. It is used togetherwith its modifications computed for various thresholdvalues of IoU (Intersection over Union, a quantity calledthe Jaccard index). Value of IoU is calculated by theformula:
IoU =Sground true ∩ Sbox
Sground true ∪ Sbox(1)
where Sground true defines the area of the etalon box,which uses to labeling of the training set, and Sbox isthe area of the detected box.
As you know, the precision is calculated by the for-mula
P =TP
TP + FP(2)
where TP and FP denote, respectively, the numberof true-positive and false-positive detection results, and,accordingly, P determines the share of correct detectionsin the total number of detections received by the neuralnetwork.
With respect to the object detection task, the numberTP determines the total number of rectangular areasfor which the value of IoU calculated with respect toground-true boxes is greater than some given threshold(usually the threshold of 0.5 is chosen). Thus, if thevalue of IoU for such a predicted region exceeds 0.5,the detection is considered to be true-positive. If thereare several detections for a given true region, then onedetection with the largest value IoU is selected, and therest are considered as FP .
The averaged value for all values of recall gives theAP :
AP =1
N
N∑
i=1
TPi
TPi + FPi(3)
where N is the number of equally spaced recall values.The value of mAP is obtained from AP by subse-
quent averaging over all available classes (for the solvedproblem of such classes two are objects ”cover” and”label”).
Results of objects detection and recognition Aftertraining the SSD-network for detection of caps and labelswe have got the results of detection with efficiencyin mAP = 0.98 (on both classes). This is the result,which in most cases guarantees an acceptable qualityof detection with the possibility of application in realsystems. Examples of detection are presented in Fig. 5.
Figure 5. Result of the detection objects by using SSD-network
SSD-network was trained during 4000 iterations, oneach of which the adjustment of network parameters was
95
made for mini-batch, with size of 8 images. Evolution ofthe mAP presents in fig. 6.
Figure 6. Evolution of the mAP during training
VI. STRUCTURE OF THE DECISION-MAKING SYSTEMKNOWLEDGE BASE
A. Representation of the trained ANN in the knowledgebase
At this stage of integration, the knowledge base storesthe trained ANNs and following data about its:
• type of ANN;• type of input data;• set of recognized classes or output type;• additional identifiers for linking objects in the
knowledge base and corresponding recognition en-gine components
At this stage, it is necessary to specify the type ofANN when there are several different ANN of a certaintype in the system and there is a need to make a decisionon the use of a particular network. The system shouldindependently make a decision on the basis of knowledgeabout the recognized object, and therefore there is a needto describe in the knowledge base the entire hierarchy ofsubject domains of ANN proposed in [1]. With the helpof this hierarchy of subject domains, it is possible todescribe such important for making decisions knowledgeabout the use of the trained ANN as:
• class of problems solved by ANN;• architecture of ANN;• the average work time of ANN;• quality of recognition;• and others.In addition, such a detailed description of the trained
ANN in the knowledge base can be used to provideinformation support to the engineer who will update thearchitecture or retrain the ANN.
A lot of IDs in an external module is used to communi-cate the results of the recognition module with fragmentsthe knowledge base. For example, it is necessary tomake a clear matching between the recognized classesin the recognition module and the corresponding classesin the knowledge base. Different recognizable classesfrom different trained ANN in the recognition module
can correspond to the same recognized class from theknowledge base. Naturally, when moving to the nextstage of integration of semantic and neural networkmodels and the implementation of neural network modelson OSTIS technology, the need for a set of integrationidentifiers will disappear, and the system is designedin advance so that the removal of this set occurs withminimal effort.
Figure 7 shows a piece of the knowledge base thatdescribes two trained ANNs, where one classifies theimage and the other classifies the text. Also, each net-work has a set of recognizable classes that intersect inthe knowledge base, but these classes are different in therecognition module.
Figure 7. Representation of trained ANNs in the knowledge base
B. Representation of logical rules for decision making
To make decisions, the system must have a set ofimplicative and equivalence tuples, which for brevity willbe called logical rules. According to these rules, thereverse inference is used to make the decision or a set ofdecisions. The rules are described in the knowledge baseusing constant and variable sc-nodes and sc-connectors.Inference works on the basis of such rules. Inferenceuses the if part as search patterns in the knowledgebase. When matching the if part of a statement wasfound, the system generates the knowledge described inthe then part of the implicative bundle of the usedlogical rule. For logical rules, presented in the formof equivalence tuples, the mechanism of its using issimilar, with the only difference that in place of the if-then parts there can be any part of the equivalence tuple.
It should be noted that the logical rule can be thespecification of agents or programs. These specificationsare represented in the form of the implicative tuple,in which if part describes the input data, and inthen part describes the output data. When making theinference, the problem solver will use these logical ruleson a par with the rest, but when using these logical rules,the appropriate agent or program will be called.
96
Each logical rule has the number of times it is used byinference. This technique will allow the system to self-learn and speed up the inference for trivial situations.
Logical rules are related to the specific subject domainin which its are used. In the case of the task underconsideration, this is the subject domain of productmarking. Figure 8 shows a fragment from this subjectdomain that describes the set of recognizable yogurtbottles of a certain class, its current standard markingof the current product that will be tested, the markingphoto and the marking recognition result by module’srecognized.
Figure 8. Fragment of the subject domains of product marking forbottles of yogurt X
The knowledge presented in figure 8 creates a pre-requisite for the use of a logical rule for checking ofmarking compliance to standard(figure 9). This rule is:matching the marking of a product from a certain subsetof the marked product with the standard marking of thissubset is equivalent to the fact that this product belongsto a set of correctly marked products. This rule should beused after the recognition is completed in the first placeto effectively handle most of the situations that arise inthe system when the products are marked correctly.
However, before using the rule shown in figure 9, it isnecessary to compare the recognized marking with thestandard, for which it is needed to use a simple programof string comparison. This program is bound to a logicalrule (figure 10), which says that if you apply to the inputof this program two strings, it will generate in memoryone of the three structures:
• strings are equal;• the first string is greater than the second in lexico-
graphical order;• the first string is smaller than the second in lexico-
graphical order.
C. Representation of the system work result
One of the most important features of the system isthe ability to explain made or proposed decisions. Forthis purpose, the inference makes a decision tree in thecourse. Decision tree stores the input recognition data,
Figure 9. Logical rule for checking of marking compliance to standard
Figure 10. Logical rule of using a program compared to two strings
the recognized marking, the chain of applied logicalrules and the applied (proposed for application by theengineer) decision.
With the help of the decision tree, it becomes possibleto restore the chain of logical rules that led to the finaldecision. The restoration of such a chain is not a trivialtask, since the use of reverse inference may lead todeadlocks after using of logical rules, and the knowledgegenerated as a consequence of using these rules may bea prerequisite for using of other logical rules which willlead to the final decision (or decisions).
VII. PROBLEM SOLVER OF DECISION-MAKINGSYSTEMS
The general problems of the decision-making system’sproblem solver are:
• access to the knowledge base;• processing (analysis, verification and generation) of
knowledge;• interaction with other modules of the system.In OSTIS technology, problem solvers are constructed
on the basis of the multi-agent approach. According tothis approach, the problem solver is implemented as a set
97
of agents called sc-agents. All sc-agents interact throughcommon memory, passing data to each other as semanticnetwork structures (sc-texts).
Initiation condition of the sc-agent is some event in theknowledge base. These events are changes of temporarynon-belonging to the temporary belonging of the elementto the situative set, which is interpreted as a set of sc-agent initiation commands. Each command represents thedata that will be processed by the sc-agent. This data canbe a single sc-element and its semantic neighborhoodavailable in common memory, or some structure (sc-structure) linked by such a sc-element [8].
After the operation started and executed, the temporarybelonging is replaced with a temporary non-belonging,however, new temporary belonging may appear that willinitiate the work of other sc-agents.
Sc-agents can be divided into external and internal.External sc-agents interact with the knowledge base butare its not part of it. Internal sc-agents are part ofthe knowledge base and can be implemented using theinternal language called SCP.
It should also be noted that some agents may be non-atomic. This means that two or more other sc-agents areused to implement its functionality.
Copies of the same sc-agent or functionally equivalentsc-agents can operate in different ostis-systems, whilebeing physically different sc-agents, since it is assumedthat the proposed system can be used in other systemsof quality control marking, and some sc-agents can beused in other decision-making systems. Therefore, it isadvisable to consider the properties and typology not ofsc-agents, but of classes of functionally equivalent sc-agents, which we will call abstract sc-agents.
A. Abstract non-atomic sc-agent of marking quality con-trol system
The whole problem solver of the system under con-sideration can be represented as a decomposition of anabstract non-atomic sc-agent of quality control markingsystem, which is presented in figure 11.
Before going to the consideration of the main sc-agent, which is an abstract non-atomic sc-agent of qualitycontrol marking system, there is a brief description of therest:
1) Abstract sc-agent of initiation the marking verifica-tion process: external agent who receives a photo of themarked product from the camera and creates a semanticstructure in the knowledge base that describes the productfor verification, a photo with its marking, as well as thesource of the photo (number of the camera from whichthe photo was taken).
The figure 8 presents an example of sc-structures,which this agent creates in the memory.
2) Abstract sc-agent of the search ANN for recogni-tion: internal sc-agent that reacts to the appearance ofa new product with a photo of marking. Since different
Figure 11. Decomposition of abstract non-atomic sc-agent of qualitycontrol marking system
ANNs(different products, cameras and shooting angles)can be used to recognize different marking sources, theagent finds a suitable ANN in the knowledge base andinitiates its using with the help of an abstract sc-agentinteraction with the recognition module.
3) Abstract sc-agent of interaction with the recogni-tion module: external sc-agent that reacts to the eventsof ANNs usage from the recognition module. Receivesthe pointer on the necessary ANN and input data. Afterfinish ANN work, it put the result of its work into theknowledge base and then initiates the work of an abstractnon-atomic sc-agent of decision-making.
4) Abstract sc-agent for interaction with the roboticinstallation control module: external sc-agent, whosetask is to make various external requests to the roboticinstallation control module, such as checking the levelof paint in the printer, changing the marking label orrejection of a certain bottle.
5) Abstract sc-agent of interaction with interface: in-ternal sc-agent, whose task is to implement user requeststo the knowledge base.
B. Abstract non-atomic sc-agent of decision-making
The main task of the abstract non-atomic sc-agent ofdecision-making is to develop and apply a decision aboutthe quality of the marking applied to the products. Inturn, this agent is decomposed into:
• Abstract non-atomic sc-agent of search decision;• Abstract sc-agent of using decision;• Abstract sc-agent of creation messages to the user;• Abstract sc-agent of providing the reason of deci-
sion.Abstract non-atomic sc-agent of search decision is
work on development of decision and compilation ofdecision tree. Its functionality is implemented with twoagents:
98
1) Abstract sc-agent of using reverse-inference: thework of this sc-agent is initiated by an abstract sc-agentof interaction with the recognition module immediatelyafter adding the recognition result to the knowledge base.Figure 8 shows an example of the initial knowledge withwhich this agent starts working.
The search for a decision is divided into two stages.At the first stage, sc-agent checks whether the productsare correctly marked. At the second stage, which beginsonly if the product is not correctly marked, there is asearch for the decision to the situation.
Since the sc-agent uses reverse inference, it starts withthe final states. At the first stage, it tries to use logicalrules before appearing one of the following semanticconstructions in the knowledge base:
• the product belongs to set of correctly markedproducts(figure 12);
• the product does not belong to set of correctlymarked products(figure 13).
Figure 12. Example of the semantic structures describing the belongingof the product to set of correctly marked products
Figure 13. Example of the semantic structures describing not belongingof the product to set of correctly marked products
At the start work of agent tries to create sets oflogical rules(by pattern search), the application of whichwill lead to the appearance of the necessary semanticconstruction in the knowledge base. Next, it tries to applythe most frequently used logical rule. The logical rulecan be applied when there is semantic construction inthe knowledge base that isomorphic to the constructionthat was obtained by substituting nodes associated withthe processed product into a template from a logical rule.This pattern is the first part of the logical rule, the secondpart describes the knowledge that will be generated afterapplying this logical rule.
If the rule can be applied, the system initiatesabstract sc-agent of using logical rule, and adds the log-ical rule and the result of its using to the decision tree.
In the case when there is not enough knowledge toapply a logical rule in the system, the agent recursively
initiates the work of itself, where it is already trying tofind logical rules, the application of which will lead to theappearance of the missing knowledge in the knowledgebase.
If the using of any logical rule does not result in theappearance of the necessary semantic constructions inthe knowledge base, the agent reports that it can’t findthe decision for this problem.
In the second stage of search system uses the sameprinciple, only the recursive search of logical rules startswith another set of final states:
• send messages to the engineer about the lack ofpaint in the printer;
• send a message to the engineer about shifting cam-era;
• request permission from the engineer to update theprinted marking;
• request permission from the engineer to update thestandard marking;
• ignore.This list can be expanded after adding new logical
rules to the system.2) Abstract sc-agent of using logical rule: sc-agent
receives the logical rule and the matching constant nodesto variable nodes of one of the logical rules parts. Next,it uses a logical rule, which is expressed in one of threeactions:
• if the logical rule is implicative or equivalencebundle, the sc-agent will substitute the matched con-stant nodes into the logical rule nodes and generatethe knowledge described in the ”to” part(for theequivalence bundle, the if-then roles are specifiedat the sc-agent input for each part of the bundle);
• if the logical rule assumes the call of any program,it is called with the matched input data, after whichthe sc-agent waits for program completion;
• if the logical rule assumes the initiation of any sc-agent, it is initiated with the matched input data,after which the main sc-agent waits for the comple-tion of the initiated sc-agent.
After applying the logical rules, the agent increasesthe use count of this logical rules and exits.
Next, let’s take a brief look at the work of other sc-agents:
3) Abstract sc-agent of using decision: the task of thissc-agent is using of the developed decision in the processof inference. The decision could be sending a message tothe engineer, reaction to the response of the engineer, theautomatic rejection of products, the change of a markinglabel, etc.
4) Abstract sc-agent of creation messages to the user:this sc-agent is initiated by the abstract sc-agent of usingdecision in the case when there is a need to compose amessage to the user whose template was defined at thestage of decision search.
99
The task of the sc-agent is to substitute the necessaryvariables into the message template. Variables can be:control point, printer, paint level in the printer, etc.
5) Abstract sc-agent of providing the reason of deci-sion: sc-agent is initiated by user request when he wantsto see the reasons for decision-making. Sc-agent receivesthe node of the decision, from which the decision tree iseasily extracted.
The task of the sc-agent is to transform the decisiontree into a view that is readable for user. To do this, thesc-agent creates a chain of logical rules from the decisiontree, the using of which led to the decision and cutsoff the logical rules, the using of which did not help inmaking the decision. Sc-agent made naturally languagetext for on the basis of natural language formulations thateach logical rule has.
VIII. EXAMPLE OF SYSTEM’S WORK
Let us consider an example of the system operation onthe case of the camera shift on the tape for marking ofsome yogurt bottles. The engineers set up the system toreceive a message with a marking problem if problemswere found with the recognition of three consecutivebottles.
The system gets to check the next bottle. System createa description in the knowledge base that includes:
• photo of the bottle cap with lint to the camera fromwhich the photo was taken;
• specify the type of product that the bottle is;• specify the control point where the photo was taken.After that, the photo of the bottle is transferred to
the recognition module to determine the bottle markingcorrectness.
Figure 14 shows a part of the knowledge base thatdescribes the knowledge about the processed bottle afterfinish work of the recognition module, which shows thatthe module was unable to recognize the marking.
After finish work of the recognition module, theproblem solver starts searching for a decision using thereverse-inference, the logic of which is described in theVII section.
First of all, the problem solver tries to use all logicalrules in which the concept of correctly marked productsis involved. The solver tries to apply the rule of checkingfor compliance with the standard marking, shown infigure 9.
Since it is not known whether the recognized markingis equal to the standard one, solver call the lexicographicstring comparison program, the specification of inputsand outputs of which are presented in figure 10.
Because the strings are not equal, the solver tries touse the following logical rule: if a product belongs toa set of consecutive unmarked products at its controlpoint and the capacity of this set is greater than theallowable number of consecutive unmarked products,
Figure 14. Fragment of the knowledge base that describes knowledgeabout the processed bottle after finish work of the recognition module
then the product does not belong to a set of correctlymarked products (figure 15).
Figure 15. Logical validation rule for exceeding the allowable numberof consecutive unmarked products
The solver can’t use this rule because there is noinformation about whether a product belongs to the setof consecutive unmarked products. The solver then looksfor logical rules that can provide this information andfinds the following rule: if the marking of a product isequal to the empty string, then the product belongs tothe set of consecutive unmarked products(figure 16).
This rule also cannot be applied, since the markingstill needs to be compared with an empty string, forwhich solver uses the lexicographical string comparisonprogram (Figure 10). Next, the solver recursively goes upthe search tree and applies the logical rule from figure
100
Figure 16. Logical rule of handling the situation of exceeding theallowable number of consecutive unmarked products
16 and then the logical rule from figure 15. The searchfor a decision is completed since the logical conclusioncame to one of the two final states of the first stage.
Since the product was not added to the set of correctlymarked products, the second stage begins. The problemsolver starts the reverse-inference, starting with one ofthe final states of the second stage described in the VIIsection.
The first logical rule that the solver will try to applyis as follows: if the paint level in the printer is zero,then the decision is to send the engineer reports aboutthe lack of paint and to call the program of productsrejection (figure 17).
The solver can’t apply this rule because the paint levelin the printer greater than zero and it finds the followingrule: if the paint level in the printer is zero, then thedecision is to send the engineer reports about the lackof paint and to call the program of the products rejection(figure 18).
After applying this logical rule, system creates thedecision node, which stores the control point, the set ofmessage templates to be sent, and the set of programsto be called for the decision making. Next, an abstractsc-agent of using decision works with this node, whichcompiles and sends all messages to the engineer, as wellas calls all the necessary programs. All the necessary pa-rameters for composing messages and calling programsare taken from the control point attached to the decisionnode.
CONCLUSION
The use of the considered approach to the buildingof decision-making systems based on the integration
Figure 17. Logical decision rule in case of paint lack in the printer
Figure 18. Logical rule of decision-making in case of camera shift
101
of neural network and semantic models allows us todesign systems with a high level of intelligence. Suchsystems are able not only to make or propose decisionsbut also to provide its justification. However, a greaterlevel of integration is required for a deeper retrospectionof the system, in which the system will be able toanalyze and justify its work not only in the searchfor decision but also on the recognition stage. Namely,the implementation of the neural network model in theknowledge base and its processing with the help of aproblem solver.
The proposed subsystem of quality control markingfor the JSC Savushkin product can be scaled to anyproduct of the company with minimal changes, it willbe enough only to configure the recognition module todetect marking on new products. The basic decision-making mechanism can be used in other decision-makingsystems, as it depends only on a set of logical rules,which is made by the engineer for each system.
ACKNOWLEDGEMENTS
The presented results were conducted in close collabo-ration of research teams of the Department of IntelligentInformation Technologies of Belarusian State Universityof Informatics and Radioelectronics, Department of In-telligent Information Technologies of Brest State Tech-nical University and JSC Savushkin product. Authorswould like to thank Daniil Shunkevich, associate profes-sor of the Department of Intelligent Information Tech-nologies of Belarusian State University of Informaticsand Radioelectronics, for help and valuable comments.
REFERENCES
[1] V. Golovko, A. Kroshchanka, V. Golenkov, V. Ivashenko, M. Kovalev,V. Taberko, and D. Ivaniuk, "Integration of artificial neural networksand knowledge bases,"Otkrytye semanticheskie tekhnologii proektirovaniyaintellektual’nykh system [Open semantic technologies for intelligentsystems], vol. 2, pp. 119–132, 2018.
[2] V. Golenkov and N. Gulyakina, "Proekt otkrytoi semanticheskoitekhnologii komponentnogo proektirovaniya intellektual’nykh sistem. Chast’2: Unifitsirovannye modeli proektirovaniya [project of open semantictechnology of component design of intelligent systems. part 2: Unified designmodels] Ontologiya proektirovaniya [Ontology of design], no. 4, pp. 34–53,2014, (in Russian).
[3] G. Zagorulko, Y. Zagorulko, "Podhod k organizacii kompleksnoipodderjki processa razrabotki intellektualnih SPPR v slaboformalizovannihpredmetnih oblastyah Otkrytye semanticheskie tekhnologii proektirovaniyaintellektual’nykh system [Open semantic technologies for intelligentsystems], vol. 2, pp. 61–64, 2016.
[4] (2018, Dec.) IMS metasystem. [Online]. Available: http://ims.ostis.net[5] D. Pospelov, "Situacionnoe upravlenie. Teoriya i praktika Moscow: Nauka,
228p, 1986, (in Russian).[6] V. Ivashenko, "Ontologicheskaya model’ prostranstvenno vremennykh
otnoshenii sobytii i yavlenii v protsessakh obrabotki znanii Vestnik BrSTU,Brest, No5(107), pp. 13-–17, 2017, (in Russian).
[7] V Tarasov, "Ot mnogoagentnih sistem k intellektualnim organizaciyamfilosofiya psihologiya informatika Moscow: Editorial URSS, 352 p., 2002,(in Russian).
[8] D. Shunkevich, “Agent-oriented models, method and tools of compatibleproblem solvers development for intelligent systems,” Otkrytyesemanticheskie tekhnologii proektirovaniya intellektual’nykh system [Opensemantic technologies for intelligent systems], vol. 2, pp. 119–132, 2018.
[9] E. Andreev, N. Kucevich, O. Sinenko, "SCADA-sistemy: vzglyadiznutri Moskow: RTSoft, 2004, 176 p.
[10] I. Davydenko, “Semantic models, method and tools of knowledgebases coordinated development based on reusable components” Otkrytyesemanticheskie tekhnologii proektirovaniya intellektual’nykh system [Opensemantic technologies for intelligent systems], vol. 2, pp. 99–118, 2018.
[11] A. Ragovskii, "Intellektualnaya mnogoagentnaya sistema deduktivnogovivoda na osnove setevoi organizacii Iskusstv. intellekt i prinyatie reshenii[Artificial intelligence and decision making], vol.2, pp. 73-86, 2011.
[12] V. Vagin, A. Zagoryanskaya, M. Fomina, "Dostovernii i pravdopodobniivivod v intellektualnih sistemahMoscow: FIZMATLIT, 2008, 704 p., (inRussian).
[13] S. Yakimchik, D. Shunkevich, "Principi postroeniya reshatelei zadach vprikladnih intellektualnih sistemahMinsk, ITS 2014, pp. 160-161. (inRussian).
[14] W., Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, A.C. Berg, "SSD: Single Shot MultiBox Detector [Online]. Available:https://arxiv.org/pdf/1512.02325.pdf.
[15] A. Wong, M. J. Shafiee, F. Li, B. Chwyl, "Tiny SSD: A Tiny Single-shotDetection Deep Convolutional Neural Network for Real-time EmbeddedObject Detection [Online]. Available: https://arxiv.org/pdf/1802.06488.pdf.
[16] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T.Weyand, M. Andreetto, H. Adam, "MobileNets: Efficient ConvolutionalNeural Networks for Mobile Vision Applications [Online]. Available:https://arxiv.org/pdf/1704.04861.pdf.
[17] S. Ren, K. He, R. Girshick, J. Sun "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks [Online]. Available:https://arxiv.org/pdf/1506.01497.pdf.
[18] J. Hui, "Object detection: speed and accuracy comparison (FasterR-CNN, R-FCN, SSD, FPN, RetinaNet and YOLOv3) [Online].Available: https://medium.com/@jonathan_hui/object-detection-speed-and-accuracy-comparison-faster-r-cnn-r-fcn-ssd-and-yolo-5425656ae359.
[19] J. Redmon, S. Divvala, R. Girshick, A. Farhadi, "You OnlyLook Once: Unified, Real-Time Object Detection [Online]. Available:https://arxiv.org/pdf/1506.02640.pdf.
[20] V. Golovko, V. Krasnoproshin, "Neirosetevie tehnologii obrabotki dannih:uchebnoe posobie Minsk: BSU, 263 p., 2017, (in Russian).
[21] K. Simonyan, A. Zisserman, "Very Deep ConvolutionalNetworks for Large-Scale Image Recognition [Online]. Available:https://arxiv.org/pdf/1409.1556.pdf.
[22] K. He, X. Zhang, S. Ren, J. Sun, "Deep Residual Learning for ImageRecognition [Online]. Available: https://arxiv.org/pdf/1512.03385.pdf.
[23] V. Golovko, A. Kroshchanka, D. Treadwell, "The Nature of UnsupervisedLearning in Deep Neural Networks: A New Understanding and NovelApproach Optical Memory And Neural Networks (Springer Link), 2016,Vol. 25, 3, pp. 127—141.
[24] J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer,Z. Wojna, Y. Song, S. Guadarrama, K. Murphy, "Speed/accuracy trade-offs for modern convolutional object detectors Computer Vision and PatternRecognition, 2017, pp. 7310–7319.
[25] J. Hui, "What do we learn from single shot object detectors (SSD,YOLOv3), FPN & Focal loss (RetinaNet)? [Online]. Available:https://medium.com/@jonathan_hui/what-do-we-learn-from-single-shot-object-detectors-ssd-yolo-fpn-focal-loss-3888677c5f4d.
[26] K. Oksuz, B. C. Cam, E. Akbas, S. Kalkan, "Localization Recall Precision(LRP): A New PerformanceMetric for Object Detection [Online]. Available:https://arxiv.org/pdf/1807.01696.pdf.
ПРИНЦИПЫ ПОСТРОЕНИЯ СИСТЕМПРИНЯТИЯ РЕШЕНИЙ НА ОСНОВЕИНТЕГРАЦИИ НЕЙРОСЕТЕВЫХ ИСЕМАНТИЧЕСКИХМОДЕЛЕЙ
Головко В. А., Крощенко А. А., Таберко В. В.,Иванюк Д. С., Ивашенко В. П., Ковалев М. В.
В работе рассматриваются преимущества интегра-ции нейросетевых и семантических моделей для по-строения систем принятия решений. Предложен под-ход к интеграции искусственных нейронных сетейс базами знаний по входам и выходам и специфи-кация этих сетей в базе знаний с использованиемонтологий соответствующих предметных областей.Предложенный подход рассматривается на реальныхпроизводственных задачах ОАО «Савушкин продукт»для контроля качества маркировки продукции.
Received 29.12.18102
Semantic analysis of voice messagesbased on a formalized context
Vadim Zahariev, Timofei Lyahor,Nastassia Hubarevich, Elias Azarov
Belarussian State University Informatics and RadioelectronicsMinsk, Belarus
[email protected], [email protected],[email protected], [email protected]
Abstract—The report is devoted to the problem of usingformalized contextual information for the semantic analysisof voice messages in conversational systems with a speechinterface. The paper proposes an approach based on severalfundamental principles: the transition from the acousticpattern to the semantic representation bypassing a separatestage of textual presentation of information, saving andprocessing contextual information at all levels in a singleknowledge base, transferring the linguistic processing stageto the semantic analysis block (which allows to take intoaccount not only statistical but also semantic links at thislevel), of applying feedback from the semantic level to thelower level to adjust the result of links work. To implementthe approach, the original signal processing technique basedon instantaneous harmonic analysis, convolutional neuralnetworks for solving the classification problem, as well asthe capabilities of the OSTIS methodology and technologywere used.
Keywords—natural language understanding, context for-malization, automatic speech recognition, neural networks
I. INTRODUCTION
The ability to conduct a dialogue with the user isone of the key and distinctive features of intelligentsystems. This process can be realized effectively onlywhen the dialogue flows in the most natural way – usingthe verbal channel of communication, i.e. through thespeech interface.
The latest achievements in the field of machine learn-ing and artificial intelligence, connected primarily withthe development of neural network approaches and meth-ods of formalisation of semantics, made it possible tobring qualitative characteristics of dialogue systems withspeech interface to the level of commercial solutions [1].This fact in it’s turn contributed to the rapid spread ofthis technology on the mass market, primarily in the formof personal voice assistants such as "Alexa" (Amazon),"Siri" (Apple), "MicroSoft", "Alice" (Yandex) [2].
An important component of the dialogue system witha speech interface is a module of recognition andcomprehension of speech signal [3], [5]. It allows todistinguish the basic semantic entities in the statement,to define relations between them, and to take into ac-count peculiarities of context. The latter possibility is of
particular importance, due to the variety of conditionsin which dialogue systems are currently used (indoorsand outdoors, in the car, in the office, etc.), which leadsto an increase in ambiguity in recognition and, as aresult, the errors are caused [4]. The use of contextualinformation (contained both in the message itself and inexternal sources – meta-information) allows to increasethe accuracy of recognition of [6], [7], [8].
In previous works, the authors addressed to the issuesrelated to the understanding of the speech signal basedon the proposed method of semantic-acoustic analysis[9]. The main motivation of the authors is an attemptto confirm or decline the following hypothesis. Sincethe textual and speech forms of presenting the messageare equivalent in terms of the message load, there isa shorter way to go from the speech signal to thesemantic presentation (“speech” – “meaning”), than athree-level scheme for translating a speech signal into arepresentation in the form of spelling text and the furtherimplementation of its semantic processing (“speech” –“text” – “meaning”). Such a transition is carried out withthe direct perception of speech by human. Accordingto the author’s opinion this approach should give anumber of advantages. For example, reduction of errorsdue to imperfection of linguistic statistical models, theability to take into account additional information that ispresent in the speech signal (intonation, pauses, acousticenvironment), but not in the spelling text in the processof semantic analysis, and vice versa, at the level oflinguistic models to take into account information aboutthe formalized representation of the context placed in theform of ontology in the knowledge base of intelligentsystem.
In this paper, obtained results and formulated ideasare used to solve a broader problem of understandingspeech fragments of a larger volume while the interactivesystem is used in the mode of transcribing information(interview, lecture, etc.). It is also potentially one of thepromising options for such systems use [2]. To solvethis problem, we may take into account the predefinedcontext (the topic of the lecture or interview), which will
103
allow us to narrow the number of possible options whileunderstanding specific terms.
II. PROBLEM OF CONTEXT CONSIDERATION
A comprehensive consideration of the context in theprocess of dialogue with a user is the key for hisstatements understanding and interpreting. The mainproblems of the current work in the subject domaininclude: identifying the topic of conversation based onthe analysis of semantic information [10], using semanticinformation to reduce recognition errors [11], trackingthe state of dialogue by means of a formalized context[12], building contextual models based on speech [13].
However, there are some problems associated with thefact that contextual information in modern systems, inour opinion, is not used in its entirety. The first partof the context (so-called linguistic context) is modelednot at the level of semantic data presentation, but at thelevel of statistical language models that do not allowfully capturing many relationships, the complexity anddiversity of contextual links, unlike semantic models, butonly reflect certain distribution of following some words(or parts of words) after others. And only the second partof the context (situational context and meta-information)is described at the level of semantic models.
There are also certain limitations connected with thestorage and processing of context information to be im-plemented [7], [8], [15]. In modern systems, dictionaries(or even ontologies) corresponding to specific topics, arestored separately from each other, meta information fromone ontology is not available for the usage in another,their number in each system is limited [15]. Thus, thecontextual information is also stored in various isolatedparts of the system, databases and knowledge bases,containing ontologies from certain subject areas. In theexisting voice assistants, they are called «abilities» or«skills» (picture 1) [2].
The dialogue system tries to determine which one ofthe all «skills» the user is currently accessing. Then itconnects the corresponding ontology and context, forexample: «search», «news», «weather», «navigation».Thus, the system becomes task-oriented, designed for aspecific subject area, and does not always allow for a«seamless» transition between different application areas.
This causes the following main problems:• Topics can strongly intersect by concepts. If we
exclude this possibility, then it is very difficult todecide where to place a specific concept. On theother hand, if we duplicate concepts in each topic,the percentage of duplications can be very large;
• Selection of a topic is rather arbitrary, which, inturn, makes it necessary to have the followingpossibilities:
– it is easy to change the boundaries of specifictopics, both with the addition of new concepts,and with the use of existing concepts;
– it is easy to add new topics, also withoutadding new concepts, and using the conceptsthat already exist in other topics;
• Since the selection of topics is rather arbitrary, itis likely that the restriction of the context to onlyone topic may turn out to be too strong restriction;a person can use terms from different areas even inthematic speech;
• Modern approaches take into account a rather lim-ited context. As a rule, they don’t consider meta-information, for working with which a commonknowledge base, accessible throughout the system,is necessary.
III. PROPOSED APPROACH
The previous work of the author [9] reported, thatin modern speech interfaces the task of meaning com-prehension is most often solved by the «bottom-up»method. Firstly, the recognition of the speech segmentsof the signal takes place, converting them into text inthe linguistic processing module. Then a recognizedfragment is transferred to the semantic module, whichis implemented separately from the linguistic. It has aknowledge base which is independent of the linguisticmodule. The information at the input of the linguisticmodule is represented as a matrix that is made ofthe recognition probability vectors of each speech flowsegment. With successful segmentation it correspondsto a phoneme, allophone, diallophone, etc. Subsequentprocessing involves building a list of meaningful sen-tences based on some grammar from these probabilityvectors. Spontaneous speech, especially in flow and natu-ral surroundings, is often agrammatic. For example, caseendings in flexive languages are most often “swallowed”,i.e. do not pronounce clearly [16]. Additionally, in theRussian language there is almost free order of words insentences. It results in not effective use of only statisticaln-gram models for this language [17]. Therefore, it isnot enough to use only one grammar, without takinginto account the context and semantic means, especiallyat the linguistic level, as it complicates the process ofunderstanding and introduces additional distortions intoit. But in traditional architecture it is not possible to solvethis problem due to the different ways of informationstoring in the linguistic and semantic processing modules(Figure 2 a).
Therefore, the approach, that is proposed in this work,implies primarily consideration of the background, con-text identification and case-role relationships, as wellas using various available meta-information (audio de-scription in tags: genre, author, speakers, recording tran-scripts), with all this information stored in a singleknowledge base. Additionally, the authors offer to usethe feedback of the semantic module with the recognitionmodule: the search list for probable words in recognition
104
Figure 1. Natural language understanding with «skills» based context formalization [14].
is updated with associative vocabulary with subsequentrecalculation of probability vectors. By repeating thecycle it is possible to achive higher percentage of correctunderstanding of the meaning.
Since the approach proposed by the authors is a rela-tively new approach to solving the problems considered,especially with regard to the systems of recognition andunderstanding of the Russian language, it is not possibleto translate a comprehensive review of the literatureon this subject. Among the existing publications, ide-ologically close to this work, are works devoted to theconstruction of direct models for transforming soundsinto words [18], end-to-end models of segmentation ofspeech parameter sequences [19], increasing the accuracyof speech recognition through semantic analysis [20].
As already was mentioned in the previous work, thelimitations of the proposed approach are homonymy andso-called information "garbage" (words that are not inthe recognition vocabulary), as well as various kinds ofinterferences, both speech and non-speech. To eliminatevarious kinds of noise and artifacts while working witha signal in the most efficient way is possible, includingthe use of more advanced models and methods of signalprocessing. In particular, in this work it is proposed touse a hybrid model of signal representation and a methodfor estimating its parameters based on instantaneousharmonic analysis.
The approach for solving the problem of resolvingparonyms and homonyms in the context of the voicemessages understanding is described in work [9]. How-ever, when analyzing a specific message, it is proposedto use not the entire knowledge base, but some part ofthe knowledge base corresponding to a specific topic orset of topics. In accordance with the approach to thedevelopment of knowledge bases, used in the frameworkof the OSTIS Technology, the knowledge base is definedby a hierarchical system of subject domains and theirrelevant ontologies [23]. Thus, the topic corresponds tothe subject domain model and the family of ontologiescorresponding to this subject domain.
Thanks to the listed components of the technology,
it becomes possible not only to solve the problemsdiscussed above, but also to get some additional benefits,namely:
• completely eliminate duplication of information(one of the fundamental principles of the SC-code);
• remove the restriction on the number of possibletopics, even for a given set of concepts;
• to be able to specify the degree (believability) of theconcept correlation to a particular topic and take thisinto account when analyzing messages;
• to be able to specify various meta-links between top-ics, for example, to indicate related topics with anindication of the closeness degree (both qualitativeand quantitative). This will allow to intellectualizethe process of choosing the most appropriate con-cepts, i.e. if the contradiction cannot be resolvedwithin the framework of one topic, then system cantry to expand the search context by related topics;
• to be able to analyze the correctness of the knowl-edge base fragments of arbitrary configuration, setcomplex rules and relationships between objects.
In addition, the SC code allows storing and specify-ing any external files in the knowledge base. Thus ananalyzed file can be specified (for example, the authorof the lecture is indicated), as a result, the system canindependently choose more or less suitable topics basedon the analysis of this specification.
IV. IMPLEMENTATION OF THE PROPOSED APPROACH
A. General system architecture
The architecture of the system implementing the pro-posed approaches is presented in the figure 2.
It is easy to see that the standard architecture of theunderstanding subsystem (2) a), in which the recognitionprocess precedes the understanding stage, includes threetypes of transformations: signal analysis with selectingbasic units of speech flow (phonemes, morphemes),linguistic processing and translation into spelling text,and only then translation into semantic models. Ap-proaches based on statistical models, that allow to take
105
Figure 2. Speech understanding system architecture: a) implementing standart approach; b) implementing proposed approach.
into account only a fraction of the possible links, areused at the stage of the linguistic processor. In theproposed architecture, the use of semantic models insteadof statistical ones permits at an early stage (when movingfrom acoustic models to immediately semantic ones) tocarry out a detailed analysis of the context, not basedsolely on statistical relationships in word sequences.
The system consists of two main parts: modules foracoustic and semantic analysis.
B. Acoustic processing part
The speech signal is fed to the input of the analysismodule, where the procedures for dividing the signal intoframes with a duration of 50 msec with 25% overlap areperformed, the signal frames are weighted by multiplyingthe current signal fragment by the Hamming window, andthe pitch frequency is searched. Next, the parameters ofthe signal model are estimated and a characteristic vector~xm is formed for the current frame, which is placed in
a sequence of similar vectors ~X ,For speech analysis it is proposed to use a model
based on a hybrid representation of speech signal withmultiband excitation, which allows the most adequaterepresentation of any fragments of the speech signal ofa different nature of sound formation [24]. Voiced andunvoiced fragments of the signal refer to separate partsof the model: periodic (harmonic) and aperiodic (noise).
s(n) = h(n) + r(n), n = 0, .., N − 1 (1)
where s(n) – input speech signal, h(n) – harmoniccomponent, r(n) – noise component of the signal, n andN – current sample number and the total duration of theanalysis frame in samples, respectively.
The harmonic component can be represented by thefollowing expression:
h(n) =
K∑
k=1
Gk(n)
C∑
c=1
Ack(n) cos
ck n+ φck(0)) (2)
where Gk – gain coefficient on the basis of the spectralenvelope, c is the number of sinusoidal signal compo-nents for each harmonic, Ac
k – instantaneous amplitudeof the c-th component and k-th harmonic, f ck and φck(0)– frequency and initial phase of the c-th component ofthe k-th harmonic, ek is the excitation signal of the kharmonic. The amplitudes AC
k are normalized in orderto provide the sum of the energy of the harmonics equalto
∑Cc=1[A
ck]
2 = 1 for k = 1, ...,K. It is easy to see thatone of the features of the models is the fact that eachharmonica is described not by one but with c sinusoidalsomponents (multiband excitation).
In this case, the aperiodic component is modeled in thewhole frequency band, as it is observed in the spectrumof the real speech signal [26]. The apperiodical compo-nent is determined, according to the expression (1), asthe signal remainder r(n) = s(n)− h(n). Model impliesthe use of signal analysis through synthesis techniqueand subtraction of the harmonic part h from the originalsignal.
106
The aperiodic component r(n) in the frequency do-main R(w) can be approximated using its spectral enve-lope and parametrized using the linear spectral frequen-cies RLSF
p = LSF (r(n)), where p is the number ofspectral envelope coefficients [27].
The estimation of parameters of the model is pro-posed to be carried out using the original method ofinstantaneous harmonic analysis (IHA), which allowsto significantly increase the accuracy of the definitionof parameters of the periodic component [28]. In con-trast to classical methods based on a short-time Fouriertransform (STFT) or the definition of the autocorrelationfunction of a signal on a short fragment, the methodin question does not impose strict limitations connectedwith observance of the stationary conditions of the signalparameters on the analysis frame. This allows to obtaina high temporal and frequency resolution of the signal,as well as a clearer spectral picture of the localizationof energy at the appropriate frequencies 3, and as aresult, to perform a more accurate estimation of the signalparameters (on average above standard methods for 10-15 %) [29].
Consequently, for one frame of signal with the num-ber m and duration N of counts the characteristicvector which includes coefficients of model xm =[Gk, A
Ck , f
Ck ,K,C, r
LSFp ] is formed. And the acoustic
image of a signal duration M is a sequence of suchcharacteristic vectors: X = (x1,x2, ...,xM)T .
In terms of signal processing, this sequence can berepresented as a analogue of the spectrogram with anextended number of parameters, where the values ofnormalized instantaneous amplitude harmonic AC
k andlinear spectral frequencies RLSF
p , characterizing the dis-tribution of energy in the periodical and the aperiodicpart of the signal respectively (which equals the areaof low and high frequencies in the Fourier spectrogram),supplemented with information about their instantaneousfrequency fCk , and energy Gk in the band (excitationsignal parameters).
In contrast to the previous work, where the fragmentof the signal was converted into a phonetic word, andthe method of comparison of the acoustic image withthe benchmark was used, which was quite applicable tothe problem of recognition with a limited dictionary, inthis work, to realize the possibilities of working withunlimited dictionary, and fragments of high-length audiorecordings, a classic approach is used based on obtaininga sequence of spelling words in sequence characteristicsignal parameters vectors. This sequence arrives at theinput of a deep neural network to solve the problemof classifying the obtained sequence of parameters andconverting it into a sequence of phonemes of units, onthe basis of which the trigram model is built a sequenceof spelling words. Linguistic modelling was partiallycarried out using the statistical models of the HMM,
and partly in the semantic processing unit. And eachof the words will be associated with some node in thesemantic network, which will later perform the procedureof linguistic modeling of the statement already takinginto account the contextual information available on boththe linguistic and semantic levels.
The neural network architecture was chosen on thebasis of the structure of the network proposed in thework [30], which is a combination of RCNN and BLSTMnetworks. This architecture has proved to be effective forsolving the problem of recognition of Russian sponta-neous speech [31], compared to the approaches based onone type of networks [32]. The architecture is presentedin the figure 4.
Features were transformed into tensors of a dimension40 × 11 and were sent to RCNN with T = 3. Then,there was a unit that was consist of two convolutionallayers with a batch-normalization and ReLU, 3×3 kernelwith padding and 1 : 1 stride. Then, convolutional layerwith 2 × 2 and 1 : 1 stride. Finally, BLSTM’s stack(three layers with 512 units in each layer) was applied.Initialization and training of the network is carried outaccording to the schemes presented in the work [33]. Forthe input feature vector, the procedure of lowering thedimension based on the principal component analysis tothe dimension of the input layer is applied. To initializethe training, limited Boltzmann machines were used. Thenetwork was trained using the criterion for minimizingmutual entropy. The implementation of the network wascarried out using Kaldi and CNTK software packagesin accordance with the methods described in the paper[30]. Two corpuses of speech phonograms were used formodel training:
• The first corpus contains about 100 hours of au-dio recordings received from the video lectures onYouTube by automatic extraction of audio tracks.For the training of models and context accountingalso both user-provided and auto-generated subtitles(automatic captions) for the extracted audio wereused. The enclosure is characterized by variabil-ity and includes recordings containing voices of80 speakers obtained in different acoustic environ-ments. The thematic area of the video was takenin accordance with the main subject domain con-sidered in the article: lecture materials, reports ofconferences from various sections of mathematicsincluding algebra, geometry, graph theory.
• The corpus of training data and test data was madeup of available «Voxforge», «SPIIRAS» and «STC»speech corpus fragments [35]. Total duration ofaudio training set was about 30 hours. The lexiconcorresponds to the common form of speech. Sincea comparatively small amount of data were avail-able for experiments with this corpus, the systemdictionary at the moment was about 1000 words.
107
Figure 3. STFT and IHA based spectrograms
Figure 4. Neural network architecture based on a combination ofRCNN and BLSTM networks [30]
The recordings were characterized by a large inter-dictatorial variability, as well as by the diversity ofthe acoustic environment.
All phonograms in both corpora were recorded at asampling frequency of 16,000 Hz, 16 bits per sample. Totest the system, 500 phonograms were selected from eachenclosure containing phrases ranging from 10 secondsto 1 minute. The remaining phonograms were used to
train the neural network. The main feature of the trainingsample preparation process was the fact that their lexicalcomposition was selected in such a way as to maximallyreuse the existing knowledge bases made on the baseof OSTIS Technology, for example, geometry and graphtheory [23], [37]. The lexical composition of the trainingset corresponded to the concepts and relations availablein the knowledge base. For example, the most frequentwords in the training sample corresponded to the mainnodes of the ontology, containing such concepts as: ge-ometric shape, point, segment, ray, line, plane, polygon,triangle, quadrilateral, etc. As a result, there was no needto carry out the procedure of forming a knowledge basefrom scratch, but it was possible to supplement it withnew concepts.
As a phoneme alphabet was chosen a set of 54phonemes: 16 vowel phonemes, 36 consonant phonemes,one phoneme for pauses and one for speech noise.This set of phonemes has been successfully used in thedevelopment of an automatic subtitle generation systemfor real-time television shows [34]. For the simulation ofvowel sounds, 6 nuclear, 4 postnuclear, 5 prenuclear and1 preprenuclear phoneme were used. Consonant soundswere modeled using 21 hard and 15 soft phonemes. Thisseparation of vowels and consonants improves the qualityof speech signal modeling, since both vowel sounds(stressed and unstressed) and consonant sounds (hard andsoft) have noticeable differences in spectral and temporalcharacteristics.
The integration of the neural and semantic parts iscarried out using the approaches presented in [36]. Fur-
108
ther processing is carried out in the semantic processingmodule.
C. Semantic processing part
The semantic processing module, in accordance withthe architecture of systems built on the OSTIS technol-ogy, includes a knowledge base, which is interpreted asa hierarchical system of subject areas and correspondingontologies, and also a problem solver, which is inter-preted as a hierarchical system of agents, managed bysituations and events in the knowledge base.
The main classes of agents that make up the task solverof the semantic processing module were considered in[9].
Let us consider several fragments of the knowledgebase of the semantic processing module, illustrating thepossibility of solving the problems formulated earlier.
Figure 5 shows an example of correlating severalconcepts to several subject domains (SD) with an in-dication of the membership believability to a particularsubject domain. As it can be seen from the example,each concept can be part of an arbitrary number ofsubject domains. At the same time, topics separationcan significantly narrow the search area when resolvingparonyms and homonyms, for example, the word «graf»in the meaning of a noble title will be considered in thelast order for a lecture on discrete mathematics, and theword «graph» in the sense of the mathematical structurewill be the last to be considered in the framework of thelecture on history.
Figure 6 shows an example of specifying meta-linksbetween subject domain. The main relations in thiscase are the relations particular subject domain andrelated subject domain. It can also indicate the degree ofcloseness, which can be taken into account, for example,when expanding the analysis context (a particular subjectdomain is automatically considered to be related with themaximum degree of closeness).
Figure 7 illustrates an example of an audio file spec-ification (lecture recording) with attribution. In turn, theauthor of the record with an equal degree of confidencecorresponds to a set of subject domains to which thelecture will most likely be devoted.
It is important to note that such characteristics as thedegree of correspondence of a concept and a subjectdomain and the degree of relationship between subjectdomains can be established both by an expert andautomatically calculated on the basis of analysis, forexample, subject texts corpus. Thus, the task of buildingknowledge base fragments corresponding to differenttopics can be significantly simplified.
V. EXPERIMENTAL RESULTS
Experiments were conducted to identify the efficiencyof proposed approach, i.e. the application of an additionalformalized context at the semantic and linguistic levels.
The Word Error Rate (WER) metric is used to evaluateperfomance. Since at the moment the performance wasevaluated only for the signal and linguistic levels process-ing modules, this type of metric (widespread for testingASR systems) was used. It represents a normalizedlevenshtein distance between two word sequences that isaveraged for all samples. The WER is defined as followsWER = (I + D + S)/N , where I – is the numberof insertions, D –is the number of deletions, S –is thenumber of substitutions, and N is the number of wordsin the reference. The speech corpus collected on thebasis of the lecture material on YouToube had the name«YtO18Trn» and «YtO18Tst» for training and testingrespectively. The speech corpus collected from fragmentsof cases «Voxforge» was called, «SPIIRAS» and «STC»for training «VoxssO18Trn» and testing «VoxssO18Tst».Two modes of work were considered, taking into accountthe semantic context in the linguistic module and withoutaccounting. The results of the experiments are presentedin Table 1.
The obtained results allow us to assert that the use offormalized context, based on the approach suggested inthe article, allows to reduce the word error rate by 5-7% on average, depending on the size and compositionof the training sample. To obtain more representativeexperimental results, it is necessary to expand the trainingsample of audio recordings and conduct additional ex-periments. For learning of deep neural networks, shellsare used that usually include from 500 to 2000 hoursof audio recordings[20], [31]. This fact also explains therelatively low percentage of recognition accuracy and theWER metric value.
VI. CONCLUSION
An approach to the problem of semantic analysis ofvoice messages with the use of formalized context isproposed. This approach involves saving and process-ing of contextual information at all levels in a singleknowledge base, transferring a stage linguistic processinginto a block of semantic analysis, using joints specificacoustical, statistical and semantic models and methods.This approach allows acheiving deduplication of contextinformation, taking a degree (believability) of the conceptcorrelation to a particular topic and take this into accountwhen analyzing messages, analyzing the correctness of
109
Figure 5. Correspondence of concepts with subject domains
Figure 6. Metalinks between subject domains
the knowledge base fragments of arbitrary configuration,setting complex rules and relationships between objects.The original system architesture developed with the helpof signal processing technique based on hybrid speechmodel and IHA, deep neural network for solving theclassification problem, as well as the capabilities ofthe OSTIS methodology and technology are used. Thisallows to reduce the word error rate by 5-7% on average.Further work will be aimed to improving the qualitycharacteristics of the proposed approach and testing iton large corpus of speech data.
ACKNOWLEDGMENT
The research presented in this paper was conducted inclose collaboration with the Department of Intelligent In-formation Technologies of Belarusian State University ofInformatics and Radioelectronics. Authors would like tothank the research group of the Department of IntelligentInformation Technologies for productive cooperation.
REFERENCES
[1] Lemley, J. Deep Learning for Consumer Devices and Services:Pushing the limits for machine learning, artificial intelligence, andcomputer vision / J. Lemley, S. Bazrafkan, P. Corcoran // IEEEConsumer Electronics Magazine. — 2017. ‘ -– Vol. 6. -– No.2. -– pp. 48-56.
110
Figure 7. An audio file specification
[2] Hoy M. B. Alexa, Siri, Cortana, and More: An Introductionto Voice Assistants / M. B. Hoy // Medical reference servicesquarterly. -– 2018. -– Vol. 37. -– No. 1. -– pp. 81-88.
[3] Schmitt, A. Towards Adaptive Spoken Dialog Systems. /A. Schmitt, M. Wolfgang. – Springer, 2012. – 251 p.
[4] MacTear, M. The Conversational Interface: Talking to SmartDevices. / M. MacTear, Z. Callejas, D. Griol. – Springer, 2016– 422 p.
[5] Pearl, C. Designing Voice User Interfaces: Principles of Conver-sational Experiences. – O’Reilly Media, 2016 – 287 p.
[6] Bellegarda, J. R. Spoken language understanding for naturalinteraction: The siri experience // Natural Interaction with Robots,Knowbots and Smartphones. -– Springer, New York, 2014. -–P. 3–14
[7] Using context information to facilitate processing of commandsin a virtual assistant. / T. R. Gruber et al. – U.S. Patent No.9,858,925. – 2018.
[8] Bringing contextual information to google speech recognition / P.Aleksic et al. // Sixteenth Annual Conference of the InternationalSpeech Communication Association. -– 2015.
[9] Zahariev, V. A. An approach to speech ambiguities eliminatingusing semantically-acoustical analysis / V. A. Zahariev, E. S.Azarov, K. V. Rusetski // Open Semantic Technologies for In-telligent Systems (OSTIS-2018). – Minsk: BSUIR, 2018. -– pp.211 – 222.
[10] Guinaudeau. C. Improving ASR-based topic segmentation ofTV programs with confidence measures and semantic relations/ C.Guinaudeau , G. Gravier , P. Sebillot // Proc. Annual Intl.Speech Communication Association Conference (Interspeech). -– 2010.
[11] ASR error management for improving spoken language under-standing / Simonnet E. et al. // arXiv preprint arXiv:1705.09515.-– 2017.
[12] Vodolan M. Hybrid dialog state tracker with asr features/ M. Vodolan, R. Kadlec, J. Kleindienst // arXiv preprintarXiv:1702.06336. -– 2017.
[13] Weigelt S. Context Model Acquisition from Spoken Utterance /S. Weigelt, T. Hey, W.F. Tichy // International Journal of SoftwareEngineering and Knowledge Engineering. -– 2017. -– Vol. 27.-– pp. 1439-1453.
[14] Sarikaya, R. The Role of Context in Redefining Human-Computer Interaction [Electronic resourse]. Access mode:https://developer.amazon.com/blogs/alexa/post/3ac41587-f262-4fec-be60-2df2f64b9af9/the-role-of-context-in-redefining-human-computer-interaction. Date of access: 10.12.2018
[15] Sarikaya, R. The technology behind personal digital assistants:An overview of the system architecture and key components. /R. Sarikaya // IEEE Signal Processing Magazine. — (2017) -–34. 1. – pp. 67-81.
[16] Goikhman, O. Ya., Nadeina, T. M. Rechevaya kommunikatsiya[Speech communication]. M.: Infra -m., 2007. – 207 p. (inRussian)
[17] Ronzhin A. L. Russian voice interface / A. L. Ronzhin, A. A. Kar-pov //Pattern Recognition and Image Analysis. -– 2007. -– Vol.17. -– No. 2. -– pp. 321-336.
[18] Direct acoustics-to-word models for english conversationalspeech recognition / Audhkhasi K. et al. // arXiv preprintarXiv:1703.07754. -– 2017.
[19] End-to-End Neural Segmental Models for Speech Recognition /Tang H. et al. // arXiv preprint arXiv:1708.00531. – 2017.
[20] Corona R. Improving Black-box Speech Recognition using Se-mantic Parsing / R. Corona , J. Thomason, R. Mooney // Pro-ceedings of the Eighth International Joint Conference on NaturalLanguage Processing. -– 2017. -– Vol. 2. -– pp. 122 -127.
[21] From training intelligent systems to training their developmenttools / V. V. Golenkov et. al. // Open Semantic Technologies forIntelligent Systems (OSTIS-2018) – Minsk: BSUIR, 2018. –pp. 81 - 98.
[22] Shunkevich, D. V. Agent-oriented models, method and tools ofcompatible problem solvers development for intelligent systems /D. V. Shunkevich // Open Semantic Technologies for IntelligentSystems (OSTIS-2018) – Minsk: BSUIR, 2018. – pp. 119 –132.
[23] Davydenko, I. Semantic models, method and tools of knowledgebases coordinated development based on reusable components/ I. Davydenko // Open Semantic Technologies for IntelligentSystems (OSTIS-2018) – Minsk: BSUIR, 2018. – pp. 99 - 118.
[24] Sercov, V.V. An improved speech model with allowance for time-varying pitch harmonic amplitudes and frequencies in low bit-rate
111
MBE coders / V. V. Sercov, A. A. Petrovsky // Proceedings ofthe EUROSPEECH 1999. -– 1999. -– pp. 1479–1482.
[25] Serra, X. A system for sound analysis / transformation / synthesisbased on deterministic plus stochastic decomposition // PhDthesis. Stanford. -– 1989. — 178 p.
[26] Stylinou, Y. Applying harmonic plus noise model in concatenativespeech synthesis / Y. Stylinou // IEEE Trans. on Speech andAudio Processing. -– 2001. -– Vol. 9, No 1. -– pp. 21–29.
[27] Aificher, E., Dzhervis, B. Tsifrovaya obrabotka signalov: prak-ticheskii podkhod. 2-e izd. [Digital signal processing: a practicalapproach. 2nd ed.] - M.: Williams, 2004. - 992 p. (in Russian)
[28] Azarov, I.S., Petrovskii, A.A. Mgnovennyi garmonicheskiianaliz: obrabotka zvukovykh i rechevykh signalov v sistemakhmul’timedia [Instant harmonic analysis: processing of sound andspeech signals in multimedia systems]. LAP Lambert AcademicPublishing, Saarbrucken. – 2011. -– 163 p. (in Russian)
[29] Azarov, E. Instantaneous harmonic representation of speech usingmulticomponent sinusoidal excitation / E. Azarov, M. Vashkevich,A. Petrovsky // INTERSPEECH 2013: proceedings of 12th An-nual Conference of the International Speech, Lyon, France, 2013.-– 2013. -– pp. 1697–1701.
[30] Deep neural networks in Russian speech recognition /Markovnikov N. et al. // Conference on Artificial Intelligenceand Natural Language. -– 2017. -– pp. 54-67.
[31] Kipyatkova I. DNN-based acoustic modeling for Russian speechrecognition using Kaldi / I. Kipyatkova , A. Karpov //InternationalConference on Speech and Computer. -– Springer, Cham, 2016.-– pp. 246-253.
[32] Kipyatkova, I. S. Raznovidnosti glubokih iskusstvennyh nejron-nyh setej dlja sistem raspoznavanija rechi [Deep artificial neuralnetworks for speech recognition systems] / I. S. Kipyatkova, A.A. Karpov // Trudy SPIIRAN. -– 2016. -– Vol. 6. -– No. 49.-– pp. 80-103. (in Russian)
[33] Medennikov I., Prudnikov A. Advances in STC Russian Sponta-neous Speech Recognition System / I. Medennikov , A. Prudnikov// Proceedings of SPECOM-2016. – 2016. – pp. 116–123.
[34] Automated closed captioning for Russian live broadcasting / K.Levin, et al. // Proc. Annual Conference of International SpeechCommunication Association (INTERSPEECH). -— 2014. -—pp. 1438–1442.
[35] Tatarinova A. Building Test Speech Dataset on Russian Languagefor Spoken Document Retrieval Task / A. Tatarinova, D. Prozorov//2018 IEEE East-West Design & Test Symposium (EWDTS). -–2018. -– pp. 1-4.
[36] Integration of artificial neural networks and knowledge bases /V. A. Golovko and et al. // Open Semantic Technologies forIntelligent Systems (OSTIS-2018). – Minsk: BSUIR, 2018. -– pp. 133 - 146.
[37] (2018, Dec.) IMS metasystem. [Online]. Available:http://ims.ostis.net/
СЕМАНТИЧЕСКИЙ АНАЛИЗ РЕЧЕВЫХСООБЩЕНИЙ НА ОСНОВЕ
ФОРМАЛИЗОВАННОГО КОНТЕКСТА
Захарьев В.А., Ляхор Т.В., Губаревич А.В., АзаровИ.С.
Доклад посвящен проблеме применения формали-зованной контекстной информации для семантическо-го анализа речевых сообщений в диалоговых системахс речевым интерфейсом. В работе предлагается подходна основе нескольких основополагающих принципов:перехода от акустического образа к семантическо-му представлению минуя отдельный этап текстовогопредставления информации, сохранения и обработкиконтекстной информации всех уровней в единой базезнаний, переноса этапа лингвистической обработки вблок семантического анализа (что позволяет учестьне только статистические но и семантические связиуже на данном уровне), применения обратной связиот семантического уровня к нижестоящем для кор-ректировки результатов их работы. Для реализацииподхода используются оригинальная техника обработ-ки сигналов на основе мгновенного гармоническогоанализа, свёрточные нейронные сети для решениязадачи распознавания, а также модели, средства иметоды технологии OSTIS.
Received 29.12.18
112
Neural network based image understandingwith ontological approach
Natallia IskraBelarusian State University
of Informatics and RadioelectronicsMinsk, [email protected]
Vitali IskraOmnigon Communications LLC
New York, NY, [email protected]
Marina LukashevichBelarusian State University
of Informatics and RadioelectronicsMinsk, Belarus
Abstract—In this paper we propose the architecture toperform a task of semantic image analysis. The approachuses the advantages of the state-of-the art deep convo-lutional neural networks for object detection and buildsthe semantic graph that represents the scene. Ontologicalsystem is used in both graph construction and modelverification. The method can be used as a part of a moreextensive intelligent system.
Keywords—image understanding, instance segmentation,object detection, ontology, semantic graph, convolutionalneural networks, semantic analysis, intelligent system
I. INTRODUCTION
The human interpretation of the image is based on aclear understanding of the meaning of both the sceneitself and its individual elements as well as seman-tic connections between these elements. So, when onan image we see a roadway along with a traffic, weconclude, that the action takes place on the road andobeys the traffic regulations. We easily select the objectsof the scene – cars, buses, motorcycles, pedestrians,traffic signs, and road markings. Paying attention tothe objects of the scene and their relative position, weunderstand the situation well. All this happens quicklyand naturally. However, for artificial vision systems, suchan interpretation is still a challenge today.
In recent years great progress has been made in thefield of image classification [1], where the task is to as-sign a label (or class) to each image. Further developmentin image analysis went in two directions:
• the improvement of the results in the field of au-tomatic detection of multiple objects in an image(identification of object labels and the locations ofthe objects) [2];
• semantic description of the image, which, given aset of objects from the image, would allow to obtaina sequence of words describing more complex con-cepts than simply listing the objects in the image [3],thus creating a text (including the one in simplifiednatural language) describing relations between theobjects in the image.
Solving the problem of understanding and interpretingimages today requires the integration of methods from
these areas [4]. Thus, in the framework of certain modernapproaches a graph model that reflects semantic relationsbetween objects is constructed based on the results ofautomatic detection [5].
A promising direction for further development in thisarea is the use of more advanced semantic means, bothfor describing the results of image analysis (objects andrelations), and directly in the analysis process. Such toolscurrently are knowledge bases and ontologies [6].
Integration of knowledge about the image and theobjects represented on it into the knowledge base willallow, on the one hand, to improve the accuracy of under-standing through the context and information available inthe knowledge base, and on the other hand, to supplementthe results of the analysis with new knowledge, that isnot clearly presented in the analysis results, but can begenerated on the basis of these results and informationfrom the knowledge base (to “discover” the image) [7].
As part of this work, an approach to semantic imageanalysis based on the integration of a model using convo-lutional neural networks and information representationand processing tools within the framework of an Opensemantic technology for intelligent systems design isconsidered.
II. IMAGE UNDERSTANDING PROBLEM
Currently, the majority of works related to imageanalysis, including semantic analysis, are devoted tosolving image recognition tasks, which involves objectdetection, classification, and sometimes building seman-tic links between objects. The result of solving thisproblem is a description of the depicted objects, whichcan be both formal and natural-language based. For theformal representation of the identified relations betweenobjects it is convenient to use models based on semanticnetworks.
However, building complex intelligent system, espe-cially autonomous one, implies the ability of not onlyprocessing the images, acquired by the system fromthe external sources, but also the ability of the systemto understand the information that can be obtained by
113
analyzing the image, i.e. integration of recognition resultsinto the knowledge base of an intelligent system forsubsequent analysis, correction, elaboration, decision-making on the basis of the information acquired, andother problems solving.
The task of image understanding can be representedby the following pipeline:
1) Detecting the objects in an image – the estimationof the regions, containing the objects, and theirclassification.
2) Building an equivalent semantic network, its anal-ysis and verification.
3) Integrating the model into the knowledge base,eliminating synonyms, resolving contradictions.
4) Supplementing the new knowledge with the infor-mation already stored in the knowledge base.
The solution to the problem of integrating new frag-ments into the existing knowledge base, identifying andeliminating synonyms and contradictions, is also carriedout in several stages and is discussed in more detail in[8].
The approach to the development of a mechanism forsupplementing knowledge obtained by a system from theexternal sources using information from the knowledgebase is discussed in [9].
Thus, this paper will focus on the first two stages, inparticular, improving the quality of image recognitionthrough the use of a priori knowledge stored in theknowledge base. Moreover, the approach proposed inthis paper can easily be integrated with the approachesconsidered in the indicated papers and used to solve theproblem of image understanding.
III. EXISTING APPROACHES ANALYSIS
To solve such a complex task as understanding andinterpreting an image, it is necessary to integrate anartificial neural network with a knowledge database [10]:
• using hybrid neural network architectures, such asconvolutional neural networks and recurrent net-works [11],
• by application of the semantic text analysis [12].The solution will naturally include object detection
(e.g. by means of convolutional neural network approach)and constructing the semantic structure in the form ofgraph.
A. Object detection
The first step in image understanding is the detectionof the objects in an image – source image processing andfeature extraction. Today in the task of object detectionthe following subtasks can be distinguished:
1) Semantic Segmentation: for each pixel in the inputimage define its category or class [13]. This problemcan be solved by means of e.g. a recurrent network [14],however, due to the large amount of the data processed
(each pixel of the image must be processed separately),this approach is very inefficient.
2) Classification and localization: determine the classof the object in the image and its exact location. This taskis now considered to be solved [15], however, since thesolution is mainly focused on the determining of the classand location of only one object, the existing effectivesolutions [16] can be applied, for example, to a part ofthe image (or the scene) with already selected regions.
3) Object Detection: determine the class and therectangular region for each of the objects in an image.As noted above, the problem can be solved by proposingthe number of regions of interest [17] and determiningwhether there is an object in the selected block and whichclass it belongs to. It is possible to use already pre-trainedmodel of the convolutional neural network [18].
4) Instance Segmentation: the task is to determineobject contours (all visible pixels) and its class on theimage with multiple objects [19]. This way it will bepossible to analyze the exact relative position of objectsin an image, including distortions and occlusions.
For further construction of the semantic network inimage analysis the solutions of the two last subtasksbased on the class of neural network models with theso-called “region proposal” (R-CNN) are most effective:
• R-CNN [20] is a sequential image processing whichgenerates a set of region proposals using a certainpre-trained convolutional neural network [21] withthe final SVM layer [22], and linear regression formore accurate region estimation.
• Fast R-CNN [23] adds the selection of regions andthe unification of all neural networks into one modelto speed up the performance.
• Faster R-CNN [24] for even greater acceleration se-lective search of regions is used, and convolutionalfeatures are shared between all parts of the network.
• Mask R-CNN [19] in contrast to previous modelsuses a binary mask to determine not only a rectan-gular region – a candidate for an object, but alsoa specific pixels that belong to an object, which,in fact, is the solution to the problem of imagesegmentation.
In many image analysis systems proposed today pre-cise image segmentation is already a good result [25].However to fully understand and interpret the imagefurther semantic analysis is necessary.
B. Equivalent semantic network constructionTo solve cognitive tasks related to image understand-
ing the designed model should reflect the connectionsand relations between objects in an image, the propertiesof these objects (qualitative and quantitative), and otherinformation.
This model can be trained in two ways:• supervised neural network learning using already
labeled graph data [5];114
• partially unsupervised learning with pruning of theleast likely relations between the objects [26].
Modern methods have a number of general limitations:
• when identifying objects and building links betweenthem one most likely option is chosen;
• only binary relations between objects are consid-ered;
• to cut off the least likely relations only neuralnetwork methods, that have a limited set of outputsand high cost of the correction (re-training), areused;
• as a consequence of what is stated above, whencutting off the least likely relations only pairsof objects are considered, while in human imageunderstanding much more complex structures ofarbitrary configuration can be analyzed.
In addition to noted above, modern approaches arefocused on solving the recognition problem and do notconsider the remaining stages of the process of imageunderstanding. This situation is partly due to the fact,that a complete solution of the problem of image under-standing implies the possibility of adjusting intermediateresults at each of the stages and returning to the previousstages, which in turn implies using a combination ofseveral approaches to information processing, as wellas the availability of universal means for representinginformation of various kinds.
The implementation of such systems on the basis oftraditional modern means is a complex task, that involvesthe combination of heterogeneous components of thesystem through software interfaces between them. Inaddition, the system constructed in this way becomesdifficult to maintain and develop and making changes toany of the modules is quite costly.
Thus the solution to the problem of image understand-ing requires a basic technological foundation that would:
• allow to integrate various models of informationprocessing on a unified formal basis, both from thefield of machine learning, and, e.g. models of logicalinference;
• ensure the unification of the representation of het-erogeneous information in the memory of a com-puter system, including both intermediate recogni-tion results and the previously accumulated knowl-edge base;
• ensure the ability of adding and adjusting the mod-els of information representation and processing(system re-training).
The standards proposed by the W3C consortium RDF[27] and OWL [28] in particular are currently widelyused as the basis for the development of knowledge basesand ontologies. However these standards have a numberof significant limitations [29], [30] on the one hand,and regulate only low-level of information representation
on the other, almost without regard to approaches toinformation processing presented in this form.
IV. PROPOSED APPROACH
In this paper it is proposed to use OSTIS Technology[31] as a formal basis for the implementation of an imageunderstanding system.
The orientation to this technology is due to the pres-ence of the following components:
• a unified version of coding the information of anykind based on semantic networks with set-theoreticinterpretation called SC-code;
• a model of an abstract semantic memory storing SC-code constructs (sc-memory) and a model of a basicmachine for processing of SC-code structures [31];
• a model of the representation of various typesof knowledge and models, methods and tools fordeveloping knowledge bases using SC-code [32];
• models, methods and development tools for hybridproblem solvers in sc-memory based on a multia-gent approach [33] that allow to integrate variousproblem-solving models, including neural networks[34] within one solver.
Thus OSTIS Technology meets the requirements forthe technological foundation necessary to implement theproposed approach to image understanding and can serveas the basis for building a hybrid system for semanticimage analysis.
Systems developed using this technology are calledostis-systems and it is assumed that the designed systemof image understanding can be further integrated as asubsystem into other ostis-system, it is also an ostis-system by itself and follows the same rules.
As mentioned earlier the focus of this paper arethe principles of image recognition with regard to theinformation previously stored in the knowledge base.Let us consider in more detail the stages of solving thisproblem in the framework of the approach proposed inthis paper.
A. Object detection
At this stage selection of objects in the image, clas-sification and building of a topological description ofthe objects (bounding boxes in the image, see Fig. 1)is performed.
For the object detection the Faster R-CNN (deepconvolutional neural network) is used. The output of thenetwork is a set of regions (coordinates of angles) and avector with class labels probabilities for each region aspresented in the table below.
B. Building object relations
In general, building of possible connections betweenobjects can be carried out in several ways:
115
Figure 1. Object detection visualization (Test image from COCOdataset [35]).
Table IOBJECT DETECTION WITH TOP-ONE CLASSIFICATION SCORES
Class ID Class Name Score y1 x1 y2 x27 car 0.9990532 65 417 196 5837 car 0.98173344 71 9 119 65
14 motorbike 0.98688495 138 4 431 61914 motorbike 0.7723496 158 49 457 24515 person 0.99547195 32 313 335 44815 person 0.9894701 54 38 393 24815 person 0.9516165 61 568 104 58815 person 0.8993606 73 316 132 34115 person 0.8546056 52 604 131 63315 person 0.76542825 60 582 102 60015 person 0.66580576 57 594 112 61115 person 0.52531904 72 287 119 309
• based on machine learning techniques without tak-ing into account the topological connections be-tween objects (e.g. using word2vec approach [12]);
• based on machine learning techniques taking intoaccount the properties of objects and their topol-ogy; this option involves the training of the neuralnetwork on manually labeled image datasets;
• based on a priori knowledge formalized in the formof ontologies stored in the knowledge base.
It is important to note that for building the knowledgebase of big volume containing this kind of information,e.g. probabilistic rules, machine learning methods canalso be used.
In the framework of this work the choice was madein favor of the second option, because it allows quicklyenough on the basis of available free-access imagedatasets to train neural network models, which makespossible to quickly create the initial configuration ofconnections between objects, that can be later adjustedusing the information from the knowledge base. At thesame time this approach, unlike the first option, allowsto take into account the properties and locations ofobjects, and also requires a relatively small expenditure
on the creating of a priori information and has a higherperformance compared with the third option.
To build initial version of the semantic model twoneural networks are trained:
• the first neural network determines for each pair ofobjects, whether they can be a subject-object pair,using the probabilities of classes (the idea is, thatperson-motorcycle are probably subject-object, andthe sky-motorcycle is probably not);
• the second neural network builds a graph for theremaining after the first step subject-object pairs,and marks edges with possible semantic relations((person, motorcycle) -> person sits on motorcycle).
To train a neural network it is necessary for eachtraining image to have a semantic graph, such as providedin Visual Genome dataset [5].
The data in Visual Genome was pre-processed andmanually labeled. The dataset consists of seven maincomponents:
• regions• objects• attributes• relations• region graphs• scene graphs• question-answer pairs.An example of the labeled data is shown in Fig. 2
Figure 2. Training data from Visual Genome dataset [5].
The initial configuration of relations at this stage isconstructed considering the most probable classes forobjects, obtained during the previous stage. For eachconstructed link there is also a probability of belonging to
116
certain classes of links (relations), from which the mostlikely relation is initially selected.
C. Model immersion and verification
At this stage the resulting semantic model is immersedin the knowledge base (at least, the merging of nodeswith the same names is performed) and verified usingthe patterns presented in the corresponding ontologies.
In general, when the number of possible classes forrecognizable objects is large, the development of suchontologies is a laborious task and can be simplifiedby automation, including using neural network models.However, at the current stage, since we are talking abouta relatively small number of classes, the development ofsuch an ontology was done manually based on expertknowledge of the subject domain.
The statements presented in the knowledge base areinterpreted by the corresponding knowledge base veri-fication agents, that are part of the problem solver ofthe image understanding subsystem. To implement thesolver, the approach developed in the framework ofOSTIS Technology proposed in relevant works, e.g. [33],is used. One of the advantages of this approach is theability to expand the range of agents, that are includedin the solver, without significant increase of the labourcost. Thus the tools of knowledge bases verification canbe constantly improved.
If in the process of verification using the currentversion of the ontology and the current set of agents, nocontradictions has been detected, the resulting semanticmodel is accepted as final, and its further immersioninto the knowledge base, recognition and merging ofsynonyms, etc. is performed. Otherwise an adjustmentstep is performed.
D. Results correction and re-verification
In case verification process detects contradictions, thesemantic model is adjusted, which in turn involves sev-eral steps:
• the fragment of the model containing contradictionsis localized; in the current version of the approach,adjustments are made only within the localizedfragment;
• another combination of relations between objectsis selected, taking into account the probabilitiesobtained in step B, the verification is repeated;
• if for the selected classes of objects it is not possibleto select a satisfactory combination of relations, thenroll back to the beginning of step B is performed andother classes are selected for one or several objectstaking into account their probabilities, after whichthe links between objects are rearranged based onthe newly selected classes.
V. SYSTEM ARCHITECTURE
As previously discussed, the solution is proposed tobe an ostis-system with corresponding architecture [36].
To improve the system performance the current im-plementation uses Pytorch and Tensorflow in the partsrelated to neural networks. Then the detection results areput into the knowledge base of the ostis-system, wherethey are verified in accordance with the routine discussedpreviously.
The proposed system architecture is summarized in theFig. 3.
A. Object detection unit
The object detection unit is built after the Faster R-CNN [24] architecture.
1) The image is passed to the feature extractionnetwork. The feature extraction is performed byone of the image classification architectures: VGG-16 [37], ResNet-101 [38] or FPN [39], – with fewof the outer layers removed.
2) The feature map (e.g. with 256 distinct features forthe VGG) is passed to the RPN (Region ProposalNetwork). The tasks of the RPN is to determinea collection of regions of interest (approx. 2000- 5000) which have a probability to contain anobject.
a) The first layer of the RPN is the 3 × 3 × 1convolution, computing the feature vector foreach 3 × 3 window of the feature map. The3×3 window in the feature map correspondsto sufficiently large local receptive field in thesource image (228× 228 for the VGG).
b) Two parallel fully-connected layers produce aregion proposition for each of the K anchorssituated at the center of each window (K = 9– pre-defined windows with fixed scale andaspect ratios):i) The layer that calculates the “objectness”
(the probability to contain an object) ofeach of the K input windows. This layeris basically doing a 2-class anchor classifi-cation: for each anchor it decides whetherthe anchor is having significant (70%)intersection with an object enclosing rect-angle.
ii) Bounding box regression layer: for eachanchor with a positive “objectness” thelayer calculates a correction to the pre-defined anchor coordinates to match themwith the actual object enclosing rectangle(the output is 4 numbers: the correctionto the x, y and width/height of the rect-angle).
3) Some of the produced regions are going to share alot of common pixels, such regions are eliminated
117
Figure 3. System architecture.
Figure 4. Classes of situations and classes of objects.
using NMS (non-maximum suppression) algorithmto reduce redundancy.
4) Each region of interest is projected to the regionof the feature map (the result of step 1).
5) Features extracted by the feature map for the regionare adjusted to match dimensions expected bythe classification network. One of the followingtechnique is used here:
a) RoiPool [23] – maximum-pooling layer withfractional stride, ensuring the expected outputdimensionality.
b) RoiCrop [40] – learnable model, that is ableto preform scaling with interpolation.
c) RoiAlign [19] – feature map scaling using bi-linear interpolation.
6) Features are passed to the classifier network todetermine an object of the region. Class list isextended with the catch-all “background” classto give the network an opportunity to reject aproposed region. For each class (except for the
“background”) the network outputs 4 numbers inaddition to class probabilities – enclosing windowdisplacement, assuming the object class. It allowsthe network to correctly detect the window coor-dinates in case of an RPN error, given that RNPcannot distinguish different classes of objects thusis unable to contain class-specific information todetermine correct object region placement.
B. Semantic analysis unitIn general, an ostis-system consists of a system model
presented using SC-code (sc-model) and an sc-modelsinterpreting platform. At the same time the sc-modelof an ostis-system may be subdivided into an abstractsemantic memory model (sc-memory), a knowledge basesc-model, and a problem solver sc-model.
Let us consider the contents of each of these com-ponents in more detail from the angle of the imageunderstanding problem.
The approach behind the OSTIS Technology frame-work is to represent the knowledge base sc-model of
118
Figure 5. Low-probability situation classes for the example domain.
the ostis-system by hierarchic system of sc-models ofthe subject domains [32] and corresponding ontologies.As a central reusable knowledge base component, theKnowledge Base Semantic Model Kernel is developedwithin the technology. The Kernel is included in the eachnewly-created knowledge base of the system and containsa number of top-level domains and ontologies.
To solve the image understanding problem, the pro-posed systems uses concepts explored in the followingKernel comprising domains:
• Subject domain of actions and tasks• Subject domain of situations and events• Subject domain of spatial entities• Subject domain of material entities• Subject domain of temporal entities• Subject domain of parameters and values
The implemented image understanding system wasdecided to be oriented towards city traffic images pro-cessing. From that followed a development of the modelfragments for the following domains describing main ob-ject classes occurring on such images, objects relations,and typical situation classes:
• Subject domain of buildings• Subject domain of living creatures• Subject domain of vehicles• Subject domain of streets and street situations
The information required on the detection result veri-fication step is specified in the ontologies correspondingto subject domains. In particular, object classes that areexpected in the situation of corresponding classes (Fig. 4)and scene classes that are improbable in the context ofthe given subject domain (Fig. 5) are specified.
In turn, the sc-model of the problem solver is in-terpreted as a hierarchical system of agents driven bysituations and events in shared sc-memory [33]. Suchagents are called “sc-agents”. Non-atomic sc-agents,that could be decomposed to a simpler sc-agents areconsidered separately. The structure fragment of a non-atomic knowledge base verification sc-agent in the SCn-code [41] is presented below.
Non-atomic sc-agent for knowledge verification<= abstract sc-agent decomposition*:• Abstract sc-agent for compliance of relations to
its domains verification• Abstract sc-agent for compliance of action
specification to its class verification• Abstract sc-agent for compliance of class
instance to class definition verification• Abstract sc-agent for validation on the base of
uniqueness statements• Abstract sc-agent for validation on the base of
statements about the impossibility of the givensituation existence
VI. EXAMPLE
Let us consider the following example. The sourceimage is shown in Fig. 6.
The object detection results are regions and objects aspresented in Fig. 7 and in the table II.
Based on the detected objects taking into accountthe coordinates of their locations a closeness graph isconstructed (Fig. 8). For clarity the probability values ofcertain objects of the corresponding classes are omitted.
119
Figure 6. The source image (from COCO dataset [35]).
Figure 7. Object detection results.
Next the relations between the neighbouring objectsare specified in the graph construction unit (Fig. 9).The semantic analysis unit based on the nature of theselected objects determines the context of the image(street, traffic).
Further the unit determines possible errors either inthe object detection or in the relations (Fig. 10).
In our example the two fragments of the resultinggraph possibly containing errors are localized. The iden-
Table IIOBJECT DETECTION RESULTS
Class ID Class Name Score y1 x1 y2 x26 bus 0.991863 12 11 326 2977 car 0.9956397 157 313 278 5087 car 0.92731386 162 441 282 5417 car 0.9228814 146 304 183 3499 chair 0.9342198 406 342 475 480
14 motorbike 0.90591204 204 39 334 17415 person 0.9992173 121 209 479 33015 person 0.9987311 131 524 322 59215 person 0.96812516 225 4 360 8115 person 0.8665556 276 597 350 639
Figure 8. Object detection results in semantic memory.
Figure 9. Semantic model with determined relations.
tified fragments correlate with the corresponding classesof incorrect constructions in the knowledge base (infigure the belonging to the indicated classes is omitted).
In this example the first of the situations is consideredincorrect, since the presence of the object of the class“chair” is unlikely in a case of the “street situation” classin accordance with the description in the framework ofthe ontology (Fig. 4).
The second case contradicts the probabilistic statementconsidered earlier (Fig. 5), that within the scope of the“road situation” there are no situations like “person-under-person” and “car-under-car” (in the current versionit is assumed that all participants in the process are onthe street level).
120
Figure 10. Semantic model with problem fragments localized.
The object detection unit (or the graph constructionunit, depending on the nature of the error) re-checks theresults for “suspicious” regions (Fig. 11). The model isadjusted (Fig. 12).
Figure 11. Object detection for “suspicious” regions.
According to the adjusted model, taking into accountthe subject-object language relationships and replacingthe relations designation with appropriate language con-structs (in this case, verbs), the following semanticdescriptions can be constructed:
“the person rides the motorbike”“the person walks the dog”
VII. CONCLUSION AND FURTHER WORK
The paper considers an approach to improving thequality of image recognition based on the integrationof neural network models and the ontological approach.The results obtained will be used further in the contextof solving the problem of image understanding.
Furthermore, as one of the ways to develop the pro-posed approach, it is supposed to use ontologies fortraining artificial neural networks on the one hand, and touse neural networks and labeled markers for automatingontologies making on the other. The combination of theseapproaches will reduce the requirements for the volumeand quality of a priori information necessary for buildingrecognition and understanding systems, and expand thescope of application of such systems correspondingly.
Figure 12. Adjusted semantic model.
ACKNOWLEDGMENTS
The research presented in this paper was conducted inclose collaboration with the Department of Intelligent In-formation Technologies of Belarusian State University ofInformatics and Radioelectronics. Authors would like tothank the research group of the Department of IntelligentInformation Technologies for productive cooperation.
REFERENCES
[1] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” inCVPR09, 2009.
[2] J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu,X. Wang, and G. Wang, “Recent advances in convolutional neuralnetworks,” Pattern Recognition, vol. 77, pp. 354–377, 2018.
[3] M. Z. Hossain, F. A. Sohel, M. F. Shiratuddin, and H. Laga, “Acomprehensive survey of deep learning for image captioning,”CoRR, vol. abs/1810.04020, 2018.
[4] J. Johnson, A. Karpathy, and L. Fei-Fei, “Densecap: Fullyconvolutional localization networks for dense captioning,” inProceedings of the IEEE Conference on Computer Vision andPattern Recognition, 2016.
[5] R. Krishna, Y. Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz,S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma, M. S. Bernstein,and L. Fei-Fei, “Visual genome: Connecting language and vi-sion using crowdsourced dense image annotations,” InternationalJournal of Computer Vision, vol. 123, pp. 32–73, 2016.
[6] T. R. Gruber, “Toward principles for the design of ontologies usedfor knowledge sharing,” International journal of human-computerstudies, vol. 43, no. 5-6, pp. 907–928, 1995.
[7] Y. Yang, Y. Li, C. Fermüller, and Y. Aloimonos, “Neural self talk:Image understanding via continuous questioning and answering,”CoRR, vol. abs/1512.03460, 2015.
[8] V. P. Ivashenko, “Modeli i algoritmy integratsii znanii na osnoveodnorodnykh semanticheskikh setei [models and algorithms forthe knowledge integration based on homogeneous semanticnetworks],” Otkrytye semanticheskie tekhnologii proektirovaniyaintellektual’nykh system [Open semantic technologies forintelligent systems], vol. 2, pp. 111–132, 2015.
121
[9] N. Hubarevich, I. Boyko, A. Semenyaka, and A. Hardzei,“Knowledge acquisition based on natural language texts,” Otkry-tye semanticheskie tekhnologii proektirovaniya intellektual’nykhsystem [Open semantic technologies for intelligent systems],vol. 2, pp. 199–206, 2018.
[10] V. A. Golovko, V. V. Golenkov, V. P. Ivashenko, V. V. Taberko,D. S. Ivniuk, A. A. Kroshchanka, and M. V. Kovalev, “Integrationof artificial neural networks and knowledge bases,” Onthology ofDesigning, vol. 8, no. 3(29), pp. 366–386, 2018.
[11] S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[12] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean,“Distributed representations of words and phrases and theircompositionality,” in Advances in neural information processingsystems, 2013, pp. 3111–3119.
[13] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutionalnetworks for semantic segmentation,” in Proceedings of the IEEEconference on computer vision and pattern recognition, 2015, pp.3431–3440.
[14] P. H. Pinheiro and R. Collobert, “Recurrent convolutional neuralnetworks for scene labeling,” in 31st International Conference onMachine Learning (ICML), no. EPFL-CONF-199822, 2014.
[15] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma,Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg,and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Chal-lenge,” International Journal of Computer Vision (IJCV), vol.115, no. 3, pp. 211–252, 2015.
[16] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitationnetworks,” CoRR, vol. abs/1709.01507, 2017. [Online]. Available:http://arxiv.org/abs/1709.01507
[17] C. L. Zitnick and P. Dollár, “Edge boxes: Locating object pro-posals from edges,” in European conference on computer vision.Springer, 2014, pp. 391–405.
[18] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferableare features in deep neural networks?” in Advances in neuralinformation processing systems, 2014, pp. 3320–3328.
[19] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” inComputer Vision (ICCV), 2017 IEEE International Conferenceon. IEEE, 2017, pp. 2980–2988.
[20] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich featurehierarchies for accurate object detection and semantic segmenta-tion,” in Proceedings of the IEEE conference on computer visionand pattern recognition, 2014, pp. 580–587.
[21] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet clas-sification with deep convolutional neural networks,” in Advancesin neural information processing systems, 2012, pp. 1097–1105.
[22] A. Sharif Razavian, H. Azizpour, J. Sullivan, and S. Carlsson,“Cnn features off-the-shelf: an astounding baseline for recogni-tion,” in Proceedings of the IEEE conference on computer visionand pattern recognition workshops, 2014, pp. 806–813.
[23] R. Girshick, “Fast r-cnn,” in Proceedings of the IEEE interna-tional conference on computer vision, 2015, pp. 1440–1448.
[24] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towardsreal-time object detection with region proposal networks,” inAdvances in neural information processing systems, 2015, pp.91–99.
[25] X. Liu, Z. Deng, and Y. Yang, “Recent progress in semantic imagesegmentation,” Artificial Intelligence Review, pp. 1–18, 2018.
[26] B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba,“Places: A 10 million image database for scene recognition,”IEEE Transactions on Pattern Analysis and Machine Intelligence,2017.
[27] (2017, Jun.) RDF 1.1 concepts and abstract syntax. [Online].Available: http://www.w3.org/TR/rdf11-concepts/
[28] (2017, Jun.) OWL 2 Web Ontology Language documentoverview. [Online]. Available: http://www.w3.org/TR/owl2-overview
[29] (2016, Sep.) Defining N-ary relations on the semanticweb. [Online]. Available: https://www.w3.org/TR/swbp-n-aryRelations/#pattern1
[30] (2016, Sep.) Representing specified values in OWL:«value partitions» and «value sets». [Online]. Available:https://www.w3.org/TR/swbp-specified-values
[31] V. Golenkov and N. Gulyakina, “Proekt otkrytoi semanticheskoitekhnologii komponentnogo proektirovaniya intellektual’nykhsistem. Chast’ 2: Unifitsirovannye modeli proektirovaniya[project of open semantic technology of component design ofintelligent systems. part 2: Unified design models],” Ontologiyaproektirovaniya [Ontology of design], no. 4, pp. 34–53, 2014, (inRussian).
[32] I. Davydenko, “Semantic models, method and tools of knowl-edge bases coordinated development based on reusable compo-nents,” Otkrytye semanticheskie tekhnologii proektirovaniya in-tellektual’nykh system [Open semantic technologies for intelligentsystems], vol. 2, pp. 99–118, 2018.
[33] D. Shunkevich, “Agent-oriented models, method and tools ofcompatible problem solvers development for intelligent sys-tems,” Otkrytye semanticheskie tekhnologii proektirovaniya in-tellektual’nykh system [Open semantic technologies for intelligentsystems], vol. 2, pp. 119–132, 2018.
[34] V. Golovko, A. Kroshchanka, V. Golenkov, V. Ivashenko, M. Ko-valev, V. Taberko, and D. Ivaniuk, “Integration of artificialneural networks and knowledge bases,” Otkrytye semanticheskietekhnologii proektirovaniya intellektual’nykh system [Open se-mantic technologies for intelligent systems], vol. 2, pp. 119–132,2018.
[35] T.-Y. Lin, M. Maire, S. J. Belongie, L. D. Bourdev, R. B. Girshick,J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick,“Microsoft coco: Common objects in context,” in ECCV, 2014.
[36] V. Golenkov, N. Guliakina, N. Grakova, I. Davydenko, V. Niku-lenka, A. Eremeev, and V. Tarassov, “From training intelligentsystems to training their development tools,” Otkrytye semantich-eskie tekhnologii proektirovaniya intellektual’nykh system [Opensemantic technologies for intelligent systems], vol. 2, pp. 81–98,2018.
[37] K. Simonyan and A. Zisserman, “Very deep convolutionalnetworks for large-scale image recognition,” CoRR, vol.abs/1409.1556, 2014.
[38] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning forimage recognition,” 2016 IEEE Conference on Computer Visionand Pattern Recognition (CVPR), pp. 770–778, 2016.
[39] T.-Y. Lin, P. Dollár, R. B. Girshick, K. He, B. Hariharan, and S. J.Belongie, “Feature pyramid networks for object detection,” 2017IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 936–944, 2017.
[40] M. Jaderberg, K. Simonyan, A. Zisserman et al., “Spatial trans-former networks,” in Advances in neural information processingsystems, 2015, pp. 2017–2025.
[41] (2018, Jun.) IMS metasystem. [Online]. Available:http://ims.ostis.net/
НЕЙРОСЕТОВОЕ РАСПОЗНАВАНИЕИЗОБРАЖЕНИЙ С ИСПОЛЬЗОВАНИЕМ
ОНТОЛОГИЧЕСКОГО ПОДХОДА
Искра Н.А., Искра В.В., Лукашевич М.М.
В настоящей работе предлагается архитектура длявыполнения задачи семантического анализа изображе-ний. Подход использует преимущества современныхглубоких сверточных нейронных сетей для обнару-жения объектов и создает семантический граф, ко-торый представляет сцену. Онтологическая системаиспользуется как при построении графа, так и приверификации модели. Этот метод можно использоватькак часть более сложной интеллектуальной системы.
Received 27.12.18
122
The building of the production capacityplanning system for the aircraft factory
Nadezhda Yarushkina, Anton Romanov, Aleksey Filippov,Gleb Guskov, Maria Grigoricheva, Aleksandra Dolganovskaya
Ulyanovsk State Technical UniversityUlyanovsk, Russia
[email protected], [email protected], [email protected],[email protected], [email protected], [email protected]
Abstract—This article describes the basic principles of buildingthe decision support system for the production capacity planningof large aircraft factory. The method for integration of the aircraftfactory information systems with the production capacity planningsystem based on ontology merging is described. The process ofmapping the database structure into the ontological representationis performed for each information system. An integrated datamodel is formed based on the ontological representations of eachinformation system database structure. The integrated model is amechanism for semantic integration of data sources. Also presentsthe method of extracting the time series from the business processesof an aircraft factory. The model of time series forecasting basedon type-2 fuzzy sets in the task of production capacity planning ispresented.
Keywords—production capacity planning, aircraft factory, on-tology merging, semantic integration, time series forecasting, type-2fuzzy sets
I. INTRODUCTION
The technological preparation of complex productionat large enterprise requires the analysis of productioncapacities. The aim is to increase the efficiency of theuse of material, technical and human resources [1]. Thecalculation of a production capacity based on a method-ology approved in the industry has many disadvantages,like not enough precision because of averaging andtroubles with adaptation to the concrete factory. Theproposed new models and algorithms allow adapting themethodology to increase the efficiency of managementat the expense of the increasing precision of forecast ofproduction processes.
The solution of these tasks allows building a uni-fied information environment for technological supportof production. The task is to balance the productioncapacity of an aircraft factory. The current approach ofmanagement is based on using a common methodologyfor a few factories approved in the industry. Methodologycontains algorithms and coefficients, accumulated fromthe statistic of production. The main disadvantage ofthis approach is a strong discrepancy between the realproduction indicators and the collected statistical data onthe concrete factory [2].
Limitations of methodology application:• the long extraction time of statistical coefficients
from production indicators;
• the impossibility of dynamic adaptation of calcula-tions into separate periods shorter than the forecasthorizon;
• the methodology does not provide for adaptation toa specific production.
By analyzing this methodology it was found out thatthe coefficients (staff time, staff performance, equipmentperformance and depreciation of equipment) are aggre-gated and averaged information from the indicators ofproduction processes. These processes are easily repre-sented by discrete time series. Using a fuzzy approachallows creating models with more options such improv-ing quality because of applying knowledge about timeseries [3], [4]. Also by analyzing production processes,it was found that this discrete interval is the one month– the minimum forecast horizon, and the time interval inwhich the indicators are unchanged.
It is necessary to take into account the existing in-formation systems of the aircraft factory that automatevarious business processes in the process of developinga production capacity planning system. Data consistencycan be achieved by integrating a production capacityplanning system with existing information environmentof the aircraft factory. Data integration means combiningdata from different data sources and providing data tousers in a unified way. The main problems of dataintegration are [5]–[8]:
1) Heterogeneity of data models.2) Autonomy and independence of information sys-
tems from each other.3) Distribution – data can be located in different
segments of a local enterprise network and/or onthe Internet.
4) Differences in data formats.5) Differences in values representation.6) Loss of data actuality by one of the sources.Thus, it becomes necessary to solve the following
methodological tasks when organizing the informationinteraction of the production capacity planning systemwith the information environment of the aircraft factory[5]–[8]:
123
1) Creating an integrated data model. The integrateddata model is the basis of a single user interfaceof the integration system.
2) Development of methods for building mappingsbetween the integrated model and models of dif-ferent data sources.
3) Integration of data sources metadata.4) Removal of heterogeneity of data sources.5) Development of mechanisms for semantic integra-
tion of data sources.
II. INTEGRATION OF INFORMATION SYSTEMS BASEDON AN INTEGRATING DATA MODEL
Linked Data methods are commonly used to solvethe methodological problem of building an integratingdata model of information systems. Tim Berners-Leeintroduced the term Linked Data [9]:
1) Uniform resource identifiers.2) The HTTP protocol is used for accessing the list
of resources.3) Standard Semantic Web technologies: RDF, OWL,
SWRL, SPARQL.4) Hyperlinks are used to identify web documents and
domain entities.The linked data principle uses standard tools and
mechanisms to determine the semantics of relationshipsbetween entities represented by data. The OWL [10]knowledge representation language is used to describethe domain entities and the relationships between them.The OWL knowledge representation language has thefollowing advantages [9]:
1) Allows linking the domain entities and documents.2) Links and relationships between domain entities
are typed.3) The unique identifier of resources allows linking
any domain entities by any relationships.4) Each domain entity is part of global metadata and
can be used as a starting point for viewing theentire data space.
5) Information from various sources can be combinedby merging a set of entities into one semanticgraph.
6) The data model structure is flexible.Thus, the OWL knowledge representation language is
used in the integrating data model as a single unifyingdata metamodel. The integrated data model based onOWL uses common dictionaries containing terms fromvarious dictionaries of external data sources.
A. Ontological data model
Ontological engineering methods are used to imple-ment the integrating data model between the data modelof production capacity planning system and data modelsof existing at the enterprise information systems.
An ontology is a model of knowledge representationof a specific domain that contains a set of definitions ofbasic concepts (classes, individuals, properties, etc.) andvarious semantic links between them. The ontology isbased on a glossary of terms that reflecting the domainconcept and a set of rules (axioms). Axioms allowcombining terms to build reliable statements about thestate of the domain at some point in time [11].
Thus, the ontology of the integrating data model is:
O = 〈C,P, L,R〉, (1)
where C = C1, C2, . . . , Cn – is a set of ontology classes;P = P1, P2, . . . , Pm – is a set of properties of ontologyclasses;L = L1, L2, . . . , Lo – is a set of ontology constraints;R is a set of ontology relations:
R = RC , RP , RL, (2)
where RC is a set of relations defining the hierarchy ofontology classes;RP is a set of relations defining the ’class-property’ ontologyties;RL is a set of relations defining the ’property-constraint’ontology ties.
B. Mapping the data model to the ontological represen-tation
At present, relational databases (RDB) are commonlyused for the realization of data models of informationsystems. RDBs contains a description of the domainin the form of related entities (tables) [12], [13]. It isnecessary to develop a method for mapping an RDBstructure into an ontological representation of a datamodel.
The relational data model can be represented as thefollowing expression:
RDM = 〈E,H,R〉, (3)
where E = E1, E2, . . . , Ep is a set of RDB entities (tables);Ei = (name,Row,Col) is the i-th RDB entity that containsthe name, set of rows and columns;Colj = (name, type, constraints) is the j-th column of thei-th RDB entity that contains properties: the name, the typeand set of constraints;H = H1, H2, . . . , Hq is a hierarchy of RDB entities in thecase of using the table inheritance function:
Hj = EiD (x)Ek, (4)
where Ei and Ek are RDB entities;D (x) is a ’parent-child’ relation between Ei and Ek;R = R1, R2, . . . , Rr is a set of RDB relations:
Rl = EiF (x)
G (x)Ek, (5)
where F (x) is an RDB relation between Ei and Ek;G (x) is an RDB relation between Ek and Ei.
Functions F (x) and G (x) can take values: U is asingle relation and N is multiple relations.
124
The following function is used to map the RDBstructure (ex. 3) to the ontological representation (ex. 1):
F (RDM,O) : ERDM , HRDM , RRDM →→ CO, PO, LO, RO,
(6)
where ERDM , HRDM , RRDM is a set of RDB entities andrelations between them (eq. 3);CO, PO, LO, RO is a set of ontology entities (eq. 1).
The process of mapping the RDB structure into anontological representation contains several steps:
1) Formation of ontological representation classes.A set of ontological representation classes C isformed based on the set of RDB entities C Ei →Ci. The number of classes of the ontological rep-resentation must be equal to the number of RDBentities.
2) Formation of properties of ontological representa-tion classes.A set of properties P of the i-th ontologicalrepresentation class Ci is formed based on theset of columns Col of the i-th RDB entity Ei
Colj → Pj . The number of properties of the i-thontological representation class Ci must be equalto the number of columns of the i-th RDB entityEi. The name of the j-th property Pj is the nameof the j-th column Colj of the RDB entity.
3) Formation of ontological representation con-straints.A set of constraints L of the properties of the i-th ontological representation class Ci is formedbased on the set of columns Col of the i-th RDBentity Ei Colk → L. The number of constraints ofthe i-th ontological representation class Ci mustbe equal to the number of constraints of the i-th RDB entity Ei. However, there are limitationsto this approach due to the difficulty of mappingconstraints if their presents as triggers or storedprocedures.
4) Forming hierarchy of ontological representationclasses.It is necessary to form a set of ontology rela-tionships RC between all the child and parentclasses corresponding to the hierarchy of RDBentities if table inheritance uses in RDB H → RC .The domain of the j-th ontological representationrelationship RCj is indicated by the reference tothe parent class Cparent. The range of the j-th ontological representation relationship RCj isindicated by the reference to the child (or a set)class Cchild.
5) Formation of relations between classes and prop-erties of classes of ontological representation.A set of ontological representation relationshipsRP is formed based on the set of columns Col ofthe i-th RDB entity Ei and the set of RDB relations
R. Two types of relationships are formed for eachj-th ontological representation property Pj :
a) The relationship ’class-property’. The domainof the ontological representation relationshipis indicated by the reference to the i-th classCi to which the j-th property belongs, andthe range to the j-th property reference Pj .
b) The relationship ’property-data type class’.The domain of the k-th ontological represen-tation relationship is indicated by the refer-ence to the j-th property Pj . The range isindicated by the reference to the l-th classCl corresponding to the l-th RDB entity El,or the reference to the m-th ontology classCm corresponding to the data type of the j-th RDB column Colj .
6) Formation of relations between properties ofclasses and constraints of properties of classes ofontological representation.A set of relations RL of ontological representationis formed based on the set of columns Col of thei-th RDB entity.The domain of the j-th ontologicalrepresentation relationship RLj is indicated by thereference to the k-th property Pk. The range of thej-th ontological representation relationship RLJ idindicated by the reference to the k-th constraintCol → RL.
Table 1 contains the description of the mapping ofthe RDB structural components with the ontologicalrepresentation entities.
Table ICOMPLIANCE OF RDB COMPONENTS AND ONTOLOGY ENTITIES
RDB component Ontology entityTable Ei Class Ci
View Ei Class Ci
RDB Data type Colj Class Ci
Table hierarchy H Relations RC
Foreign key Colj Property Pj and Relations RP
Column Colj Property Pj and Relations RP
Constraint Colj Constraints L and Relations RL
C. Formation of an integrating data modelIt is necessary to form an integrating data model based
on the ontological representations that obtained aftermapping the RDB structure of each of the integratedinformation systems into the ontological representation.The definition of an ontological system is used as aformal representation of an integrating data model [14]:
O∑= 〈OMETA, OIS ,M〉, (7)
where OMETA is the integrating data model ontology (metaon-tology);OIS = OIS
1 , OIS2 , . . . , OIS
g is a set of ontological represen-tations of information systems that must be integrated;M is a model of reasoner.
125
The following steps are necessary to form an in-tegrating data model based on the set of ontologicalrepresentations of the information systems that must beintegrated:
1) Formation of the universal concept dictionary forthe current domain.The process of forming an integrating data modelOMETA is based on the presence of commonterminology. Ontological representations of all in-formation systems that must be integrated OIS
should be built from a single concept dictionary.The concept dictionary is formed by the expertbased on the analysis of the obtained ontologicalrepresentations.
2) Formation an integrating data model OMETA.At this step, the set of top-level classes CMETA areadded to the integrating data model OMETA. Theset of top-level classes CMETA describes systemsthat must be integrated and is used as the basis forontology merging.
3) Formation of class hierarchy of integrating datamodel OMETA.At this step, the integrating data model establishesa correspondence between the class hierarchiesCOIS
i of ontological representations OIS of infor-mation systems that must be integrated.
4) Formation of class properties of the integratingdata model OMETA.At this step, the integrating data model establishesa correspondence between the properties POIS
i
of ontological representations OIS of informationsystems that must be integrated. The expert decideswhich class properties of ontological representa-tions OIS should be included in the integratingdata model OMETA.
5) Formation of axioms of classes and properties,checking the integrating data model OMETA forconsistency.At this step, constraints LOIS
are applied to theproperties POIS
and classes COIS
of the integrat-ing data model OMETA based on the constraintspresents in the ontological representations OIS .After that, the resulting integrating data modelOMETA should be checked for internal consis-tency using the reasoner M . However, the devel-opment of methods for checking the conditions ofconstraints is required, since the existing reasonersdo not support working with such objects.
III. TYPES OF EXTRACTED TIME SERIES OF FACTORY
The task is to extract changes in the values of produc-tion processes indicators. Time series models are used fortracking these changes. The methodology for calculatingof production capacity uses some coefficients, definedabove. But these coefficients not always must be given
by an expert or a method. Each of them can be extractedon the factory. As an example, staff time can be trackedfor each factory unit; depreciation of equipment can becalculated based on summarizing volumes of completedworks.
We extract the following types of time series:• staff work time fund (fluctuating time series);• tool work time fund (fluctuating time series);• performance ratio (growing time series);• area usage (growing time series);• depreciation of equipment (growing time series).These types of time series may be different for differ-
ent factory units. For all types of processes can be iden-tified monthly indicator values. Very important to findthe following characteristics of time series: seasonality,local and global tendencies. The proposition is to useseveral models for smoothing, extracting and forecastingtendencies and values of the time series of productionprocesses.
IV. DEFINITION OF TYPE-2 FUZZY SETS TO USE INTIME SERIES MODELS
The tasks of time series modeling are solved by alarge number of methods. These methods have a differentmathematical basis, are divided according to applicationpossibilities (that is, they may have particular applicabil-ity conditions depending on the type of problem beingsolved and the nature of the time series), they mayrequire constant or temporary use of the analyst directlyduring the modeling process. An important condition forthe application of methods is the focus on obtainingshort-term forecasts. It follows from the recent featuresof the processes for which time series models are applied.
The nature of fuzzy time series due to the use of expertestimates, the inherent uncertainty of which belongsto the class of fuzziness. Unlike stochastic uncertainty,fuzziness hinders or even excludes the use of statis-tical methods and models, but can be used to makesubject-oriented decisions based on approximate humanreasoning. The formalization of intellectual operationsthat simulate human fuzzy statements about the stateand behavior of complex phenomena, forms today anindependent area of applied research, called ”fuzzy mod-eling” [15].
This direction includes a complex of problems, themethodology for solving which is based on the theoryof fuzzy sets, fuzzy logic, fuzzy models (systems) andgranular calculations. In 1975, Lotfi Zadeh presentedfuzzy sets of the second order (type-2) and fuzzy setsof higher orders, to eliminate the disadvantages of type-1 fuzzy sets. These disadvantages can be attributed to theproblem that membership functions are mapped to exactreal numbers. This is not a serious problem for manyapplications, but in cases where it is known that thesesystems are uncertain.
126
The solution to the above problem can be the useof type-2 fuzzy sets, in which the boundaries of themembership areas themselves are fuzzy [16].
It can be concluded that this function represents afuzzy set of type-2, which is three-dimensional, and thethird dimension itself adds a new degree of freedomto handle uncertainties. In [16] Mendel defines anddifferentiates two types of uncertainties, random andlinguistic. The first type is characteristic, for example, forthe processing of statistical signals, and the characteristicof linguistic uncertainties is contained in systems withinaccuracies based on data determined, for example,through expert statements.
To illustrate, note the main differences between type-1fuzzy sets and type-2 fuzzy sets. Let us turn to 1, whichillustrates a simple triangular membership function.
Figure 1. The type of fuzzy sets of the 1st (a) and the 2nd (b) types.
Fig. 1 (a) shows a clear assignment of the degreeof membership. In this case, to any value of x therecorresponds only one point value of the membershipfunction. If you use a fuzzy membership function of thesecond type, you can graphically generate its designationas an area called the footprints of uncertainty (FOU).In contrast to the use of the membership function withclear boundaries, the values of the membership functionof type 2 are themselves fuzzy functions.
This approach gave the advantage of approximating afuzzy model to a verbal one. People can have differentestimates of the same uncertainty. Especially it concernsestimated expressions. Therefore, it became necessaryto exclude a unique comparison of the obtained valueof the degree of the membership function. Thus, whenan expert assigns membership degrees, the risk of erroraccumulation is reduced because of the non-inclusion ofpoints located near the boundaries of the function andunder doubt.
V. TIME SERIES MODEL BASED ON TYPE-2 FUZZYSETS
Time series modeling based on type-2 fuzzy setsallow to build the model reflecting uncertainty of thechoice of values of coefficients or values of indicatorsdetermined by an expert. Choose an interval time seriesas type of time series for the object of modeling. For oursubject area, previously selected time series of indicatorsare easily represented by proposed type of time series:
most time series have a rare change in values. Canmark stability of intervals. For interval time series, analgorithm for constructing a model is described in [17].
The formal model of the time series:
TS = tsi, i ∈ N, (8)
where tsi = [ti, Bti ] is an element of the time series at themoment of time ti and a value in the form of a type-2 fuzzyset Bti . For the entire time series, the universe of type-2 fuzzysets is defined as U = (B1, ..., Bl), Bt−i ∈ U, l ∈ N, l - thenumber of fuzzy sets in the universe. A set Bti is a type 2fuzzy set, therefore, a type-1 fuzzy set is assigned to it as avalue. For interval time series, a prerequisite for creating type-1 sets is a part separated from the source series, limited, forexample, by a time interval of 1 day, 1 month or 1 year. Forthe selected interval, a universe of type-1 fuzzy sets is defined.
The algorithm for constructing a model will be usedthe same as described in [17], except for the moment ofchoice of intervals: they will be determined based noton the time characteristic, but on the boundaries of theinitially formed sets of type 2.
The form of fuzzy sets is proposed to use a triangulardue to the small computational complexity when con-ducting experiments.
VI. EXPERIMENT
The experiment plan implies the construction of timeseries models and the assessment of their quality. Forexperiments, time series have been generated. The fore-
Figure 2. Smoothing the time series of the coefficient
casting process at this stage will not be carried out;therefore, an internal measure of the quality of the modelwill be assessed using the SMAPE criterion [18]:
SMAPE =100%
n
n∑
t=1
|Ft −At|(|At|+ |Ft|)/2
(9)
Consider the process of smoothing the coefficient. Theoriginal time series has 60 points. For comparison, thegraph of Fig. 2 shows the smoothing of the time seriesby the F-transform method [19].
For smoothing, a set of 15 type-2 fuzzy sets and 5 setsof type-1 was selected. As can be seen from the, Fig. 2,5 points of a smooth series were obtained. SMAPE scorefor both types of smoothing:
• for F-transform - 2.01%,• for type-2 fuzzy sets - 0.65%.
127
Figure 3. Smoothing the time series of employee count
Next smooth employee count time series, Fig. 3. Forsmoothing, a set of 15 type-2 fuzzy sets and 5 sets oftype 1 was chosen. For the time series, 5 points of asmoothed series were also obtained. SMAPE score forboth types of smoothing:
• for F-transform - 47.54%,• for type-2 fuzzy sets - 13.23%.
It was also a comparison of the internal measures of thequality of the model for SMAPE with simple exponentialsmoothing. The estimates showed the best by 0.1%smoothing quality by the method we proposed usingusing type-2 fuzzy sets.
CONCLUSION
This article presents the implementation of the methodof integrating the information systems of the aircraft factorywith the production capacity planning system. The principlesof linked data and ontological engineering allows mappingdatabase structure of each information system that must beintegrated into ontological representation. From the proposedmethodology, an integrated data model is formed based onthe obtained ontological representations for each informationsystems that must be integrated.
The analysis of existing algorithms, data and informationsystems has shown a strong accumulation of errors in process ofproduction capacity planning. These principles allow improvingthe quality of technological preparation of complex industries.Proposed methods of prediction of time series are improve thequality of management decisions.
Successfully applied an approach based on type-2 fuzzy sets,to form a model of a time series of production processes. Itshould be noted that the approach based on modeling intervaltime series gives a positive result. This moment is fixed asa result of the smoothing procedure, when the number ofselected points and their values are as close as possible to thestabilization intervals.
REFERENCES
[1] N.G. Yarushkina, V.N. Negoda, Yu.P. Egorov, V.S. Moshkin, V.V. Shishkin,A.A. Romanov, E.N. Egov Modelirovanie processa tekhnologicheskoj pod-gotovki proizvodstva na osnove ontologicheskogo inzhiniringa [Modelingthe process of technological preparation of production based on ontolog-ical engineering]. Avtomatizaciya processov upravleniya [Automation ofmanagement processes], 2017, vol. 4, pp. 94-100.
[2] N.G. Yarushkina, T.V. Afanasyeva, V.N. Negoda, M.K. Samokhvalov, A.M.Namestnikov, G.Yu. Guskov, A.A. Romanov Integraciya proektnyh dia-gramm i ontologij v zadache balansirovki moshchnostej aviastroitel’nogopredpriyatiya [Integration of design diagrams and ontologies in the objec-tive of the balancing of the capacity of the aviation-building enterprise].Avtomatizaciya processov upravleniya [Automation of management pro-cesses], 2017, vol. 4, pp. 85-93.
[3] Novak, V., Stepnicka, M., Dvorak, A., Perfilieva, I., Pavliska, V. Analysis ofseasonal time series using fuzzy approach. Int. Journ. of General Systems,2010, vol. 39, pp. 305-328.
[4] Novak, V. Mining information from time series in the form of sentences ofnatural language. International Journal of Approximate Reasoning, 2016,vol. 78, pp. 1119-1125.
[5] M.R. Kogalovsky Metody integracii dannyh v informacionnyh sistemah[Methods of data integration in information systems] Available at:http://www.ipr-ras.ru/articles/kogalov10-05.pdf (accessed 06.12.2018).
[6] A.A, Kusov Problemy integracii korporativnyh informacionnyh sistem[Problems of integration of corporate information systems] Availableat: https://cyberleninka.ru/article/n/problemy-integratsii-korporativnyh-informatsionnyh-sistem (accessed 06.12.2018).
[7] O.A. Morozova Integraciya korporativnyh informacionnyh sistem [Integra-tion of corporate information systems]. Moscow, Finansovyj universitet,2014. pp. 8-23.
[8] D.U. Stepanov Sposoby integracii dannyh korporativnyh informacionnyhsistem [Ways to integrate corporate information systems data]. Moscow:Moscow, 2014. pp. 207-213.
[9] C. Bizer, T. Heath, T. Berners-Lee Linked Data – The Story So Far. Avail-able at: http://tomheath.com/papers/bizer-heath-berners-lee-ijswis-linked-data.pdf (accessed 05.12.2018).
[10] OWL 2 Web Ontology Language Document Overview (Second Edi-tion). Available at: https://www.w3.org/TR/owl2-overview/ (accessed05.12.2018).
[11] T. Gruber Ontology. Available at: http://tomgruber.org/writing/ontology-in-encyclopedia-of-dbs.pdf (accessed 05.12.2018).
[12] D. Calvanese, B. Cogrel, S. Komla-Ebri, R. Kontchakov, D. Lanti,M. Rezk, M. Rodriguez-Muro, G. Xiao Ontop: Answering SPARQLQueries over Relational Databases. Available at: http://www.semantic-web-journal.net/system/files/swj1278.pdf (accessed 05.12.2018).
[13] A. Poggi, D. Lembo, D. Calvanese, G. De Giacomo, M. Lenzerini, R.Rosati Linking data to ontologies Data Semantics, 2008, pp. 133-173.
[14] T.A. Gavrilova, V.F. Horoshevsky Bazy znanij intellektual’nyh sistem [Theknowledge bases of intelligent systems]. SPb.: Piter, 2000, pp. 59–98.
[15] Perfilieva I., Yarushkina N., Afanasieva T., Romanov A. Time seriesanalysis using soft computing methods. International Journal of GeneralSystems, 2013, vol. 42:6. pp. 687-705.
[16] J. M. Mendel, R. I. B. John: Type-2 Fuzzy Sets Made Simple, IEEETransactions on Fuzzy Systems, 2002, vol. 10/2, pp. 117-127.
[17] Narges Shafaei Bajestani, Assef Zare Forecasting TAIEX using improvedtype 2 fuzzy time series. Expert Systems with Applications, 2011, vol. 38,no. 5, pp. 5816-5821.
[18] SMAPE criterion by Computational Intelligence in Forecasting (CIF).Available at: http://irafm.osu.cz/cif/main.php (accessed 05.12.2018).
[19] Perfilieva, I. Fuzzy transforms: Theory and applicatios. Fuzzy Sets andSystems, 2006, vol. 157, pp. 993-1023.
ПОСТРОЕНИЕ СИСТЕМЫ БАЛАНСАПРОИЗВОДСТВЕННЫХМОЩНОСТЕЙ
АВИАЦИОННОГО ЗАВОДА
Н. Ярушкина, А. Романов, А.Филиппов,Г. Гуськов, М. Григоричева, А. Долгановская
В данной статье описаны основные принципы построениясистемы поддержки принятия решений для планированияпроизводственных мощностей крупного авиационного за-вода. Описан метод интеграции информационных системавиационного завода с системой баланса производственныхмощностей на основеметода слияния онтологий. Для каждойинформационной системы выполняется процесс отображе-ния структуры базы данных в онтологическое представле-ние. Интеграционная модель данных формируется на осно-ве онтологических представлений структуры базы данныхкаждой информационной системы. Интеграционная модельпредставляет собой механизм семантической интеграцииисточников данных. Также представлен метод извлечениявременных рядов из бизнес-процессов авиационного завода.Предложена модель прогнозирования временных рядов взадаче планирования производственных мощностей на ос-новании нечетких множеств типа 2.
Received 27.12.18
128
Dynamic integrated expert systems: automatedconstruction features of temporal knowledge
bases with using problem-oriented methodologyRybina G.V., Sorokin I.A., Sorokin D.O.
National Research Nuclear University MEPhI (Moscow Engineering Physics Institute)Moscow, Russian Federation
Abstract—The work is focused on the problems of inte-gration in the dynamic integrated expert systems (IES) ar-chitecture in the context of integration paradigm of artificialintelligence with models, methods and tools from other do-mains. These systems are developed basing on the problem-oriented methodology and the AT-TECHNOLOGY Work-Bench. In this paper models, methods and software, forimplementing a combined method of acquisition temporalknowledge from various sources of knowledge (experts,natural language texts, databases) are considered.
Keywords—dynamic integrated expert systems, problem-oriented methodology, AT-TECHNOLOGY WorkBench,temporal knowledge bases, integration, temporal knowledgeacquisition, temporal knowledge representation, temporalprocessing, simulation
I. INTRODUCTION
At educational-scientific laboratory "Intelligent systems andtechnologies" of the Department of Cybernetics of NRNUMEPhI are developing dynamic integrated expert systems (IES)with AT-TECHNOLOGY — software platform that implementsproblem-oriented methodology [1], [2]. This platform supportsand automates processes of prototyping and maintaining IESthroughout their lifecycle. For a number of criteria (suchas knowledge representation models, reasoning tools, object-oriented design support, etc.) AT-TECHNOLOGY is compa-rable to G2 (Gensym corp., US) [3] — leading softwareplatform for real-time expert systems. Considering the built-in subsystem of outer world simulation, AT-TECHNOLOGYeven goes ahead of G2. While G2 and some other toolslack automated knowledge acquisition, out platform offersoriginal hybrid knowledge acquisition tools enabling fuzzyand temporal knowledge acquisition from various sources Thedynamic version of the AT-TECHNOLOGY WorkBench thatsupports problem-oriented methodology for building dynamicIES is being actively developing (described in [1], [2], [4], andothers.).
II. SOME INTEGRATION FEATURES OF MODERN IESIN THE DYNAMIC IES ARCHITECTURE
In the context of solving the modern IES constructionproblems (in particular for the control of complex discretesystems), problem-oriented methodology [1], which is concep-tually related set of models, methods, algorithms and standardprocedures to create applied IES of different typologies andlevel of complexity, has the following properties [1], [2]: a pow-erful combination method of acquiring knowledge that supportsthe automated process of acquiring knowledge from the sourcesof knowledge of different typology (experts, databases, texts) is
used to gain knowledge; generalized knowledge representationlanguage designed for building models of problem areas indynamic IES allows to represent temporal knowledge, based ona modified interval Allen logic [5] and time control logic [6],together with the basic knowledge, including those containingknowledge with uncertainty, imprecision and vagueness; sup-ports the use of various output means (universal AT-SOLVERand a specialized temporal solver designed for dynamic tasks);in the context of enhanced functionality and principles of thecomponents IES deep integration provides the possibility ofimplementing simulation techniques for modeling the externalenvironment and how to interact with them: the high effi-ciency of the a large number of applied IES development,including dynamic areas of concern;instrumentally supportedby a modern software such as WorkBench (complex AT-TECHNOLOGY).
It should be noted that the conceptual basis of this method-ology is a multi-level integration into the IES processes model,modeling specific types of tasks, relevant traditional expertsystems technologies, methods and ways of building softwarearchitecture IES and its components at every level of integra-tion, etc. In particular, in the dynamic IES important place isgiven to the integration of methods and means of temporalinformation presentation and processing with the methods andmeans of the outside world simulation, in this case discretecomplex technical systems(CTS) in real time. This leads toexpansion of the intelligent control systems architecture, builton the concept of dynamic IES relevant subsystems adequatelyreflecting all the processes and laws of functioning simulatedsystems, as an integral phase of building dynamic IES.
Using [1], [2], let’s take the example of a integration modelcomponents of simulation discrete CTS models in languageRAOAT [2] with the other components of dynamic IES
Language RAOAT is used here for a description of discreteCTS simulation models. This language was developed on thebasis of well-known Russian method of RAO (Resources, Ac-tions, Operations), [7] which allow to maximize fully reflect thebehavior of any discrete type CTS (change of state resources,the emergence of regular and irregular events, and so on.)
It is important that the description of the CTS resources anddescription of the objects in the problem areas of the developedlanguage of knowledge representation are conceptually close,allowing the temporal solver at AT-TECHNOLOGY complexto use parameters of the resources from the simulation modelof discrete CTS through the working memory in the temporalwithdrawal. So are all the prerequisites for the integration ofsimulation technology with dynamic IES technology in modernintelligent control systems and etc.
129
III. AUTOMATED ACQUISITION, REPRESENTATIONAND PROCESSING OF TEMPORAL KNOWLEDGE IN
DYNAMIC INTEGRATED EXPERT SYSTEMS
Acquisition, representation, and processing of temporalknowledge (i.e., knowledge considering time as an entity ofa problem domain) play important role in the context of thesystematic approach to development of dynamic integratedexpert systems. In [1], [2] we have described basic mod-els of knowledge representation and inference tools of AT-TECHNOLOGY AT-SOLVER.
Here we consider temporal aspects of inference perfomed onknowledge bases containing some unreliable knowledge, i.e.,knowledge with such negative factors as uncertainty, inaccu-racy, fuzziness, and with constraints on variables. Generalizedmodel of temporal inference with production rules for dynamicintegrated expert systems involves processing of knowledgewith temporal dependencies together with basic knowledge ofthe problem domain.
We see the purpose of temporal inference in constructionof the event flow model interpretation and in generation ofa list of controlling actions for the problem domain. So weapplied Allen’s logic [5] with some enhancements together withOsipov’s logic of control over time [6]. We define global eventflow model by a set of temporal objects (events and intervals).Local event flow models in rules are defined by formulas ofAllen’s logic. Event flow model interpretation may be given asa set of timestamps of events and intervals.
To represent temporal knowledge in dynamic integratedexpert systems we enhanced knowledge representation lan-guage of AT-TECHNOLOGY — AT-KRL [8]. Now it allowsrepresenting temporal knowledge together with basic knowl-edge including knowledge with uncertainty, inaccuracy, andfuzziness. To do so, we introduces new basic types of objects:events and intervals; new type of object properties: conditionof event occurrence.
We modified the structure of rule antecedents: we addedlocal event flow model requirements. At last, Osipov’s controlover time concepts lead to adding new rule types into AT-KRL:reactions, aimed to provide quick response to certaion, usuallyurgent, events in probles domain,and periodic rules, aimed totrack certain duty cycles.
Reaction rules generally correspond to enhanced Allen’slogic. Their antecedents are formulas where each operand isa single temporal object ( an event or an interval). Antecedentsof periodic rules contain extra condition with firing period.These enhancements of AT-KRL allow us to describe temporalrelationships of objects in a problem domain by rules. Decisionmaking is now performed taking into account actual event flowof the problem domain.
As for inference process the major changes were made inmatching procedures: we implemented evaluation of formulascontaining temporal arguments in rule antecedents and con-struction of event flow interpretation on each inference loop.When forming event flow interpretation, events and intervalsare bound to the time axis by identifying the facts of theiroccurrence and considering the history of events. Processingof temporal parts of antecedents uses the results of eventflow interpretation construction. For active rules AT-SOLVERmatches local event flow models with constructed event flowinterpretation.
Thus, the synergy of AT-SOLVER and temporal tools ad-dresses both static and dynamic domains. Note that complexdiscrete systems produce input data for temporal inference indynamic integrated expert systems. Issues related to models,methods, algorithms, and software for simulation modeling areconsidered in a number of papers, for example [8], [9].
In expert systems development we can automate experts’work by implementing methods and tools for detecting andextracting temporal knowledge from natural language texts(NL-texts)(Text Mining and Natural Language Processing) andfrom databases (Data Mining and Knowledge Discovery inDatabases). In world practice there is a number of approaches totemporal dependencies acquisition but most of them are focusedon processing of English-language texts. Moreover, they do notconsider obtaining temporal knowledge for temporal knowl-edge bases for dynamic intelligent systems and for dynamicintegrated expert systems in particular.
The combined method of knowledge acquisition (CMKA)[1], [2] has proven its efficiency in development of static in-tegrated expert systems with AT-TECHNOLOGY. It automatesinterviewing of experts using natural sublanguage (businessprose style), data mining, and verification of knowledge bases.In [2], [4] we described the client-server architecture andtools for knowledge acquisition from geographically distributedsources of knowledge of various types: experts, natural lan-guage texts, databases.
In dynamic integrated expert systems methods of automateddetection of temporal knowledge remain an unexplored prob-lem. In particular, for extracting information about time fromtexts in Russian, only the few approaches are proposed thatpertly help to automate these processes, e.g. [10], [11]. There-fore, we focused on further evolution of combined method ofknowledge acquisition and especially its temporal enhancementby developing new methods and tools for automated construc-tion of temporal knowledge bases in dynamic integrated expertssystems.
Our approach to knowledge acquisition (directly from ex-perts by automated inter-viewing) bases on original techniqueof using patterns for solving typical problems [1]. We haveput meta knowledge about strategies of solving into heuristicsolving patterns for specific cases: diagnostics, engineering,planning, control, learning, and some other. To support thesesolving patterns we developed a number of methods and toolsfor modeling dialog scenarios used in interviewing. Thesemethods [1] address both thematic structures of dialog, i.e.problem solving pattern, and local structure of a dialog, i.e.dialog steps –specific actions and reactions between n the expertand the system.
As computer-aided interviewing of an expert goes on, theproblem solving pattern fills with structured data that can beexported to knowledge base. To derive the “action-reaction”model of dialog we use several techniques, e.g., simulation ofconsultation. Interviewing of experts is carried out automati-cally by dialog scenario interpreter.
The interpreter also generates dialog screens for enteringanswers and data including such things as uncertainty, impreci-sion, fuzziness. The specialized linguistic processor and s assetof dynamically replenished dictionaries [1] support knowledgeacquisition process. Computer-aided interviewing of expert,natural language processing, data mining from databases aretightly coupled in AT-TECHNOLOGY [2], [4], [8], [9].
We developed a technique of detection and interpretationof the simplest temporal pointers ( i.e., independent individualwords and phrases denoting time) within a single sentence.We used generic classification of temporal pointers presentedin [10], [11] together with vocabulary of Russian-languagelexemes indicating temporal relations. To model dialogs weused business prose linguistic model for medical diagnosticsand specialized linguistic processor [2].
We adopted the Random Forest algorithm [12] to workwith databases containing temporal data [13]. The ensembleof decision trees is constructed in accordance with the basic
130
Random Forest algorithm. We use multidimensional featurespace, one of which is the timestamp. However, the calculationmethod of the partitioning criterion value has changed to thearithmetic mean of entropy values. Also, the construction ofthe tree is carried out until all the elements of the subsampleare proceeds without using cut – off procedure [4], [9].
To convert the ensemble of decision trees to knowledge baseformat we use some helper tools: The main object containingall features of the feature space as attributes; The counter tomeasure time; Vote counters. Each leaf is converted to a rule ofthe following form: f the duration of all intervals correspondingto vertices on the path to the root is greater than zero, thenincrement the vote counter for the class that corresponds to thecurrent vertex.
When we extract knowledge containing temporal data fromvarious sources of different types (experts, text, and databases), we get multiple fragments containing objects, types, and rules.To merge all the fragments of knowledge together we use meth-ods, algorithms, and software tools of the combined method ofknowledge acquisition from distributed sources taking into ac-count temporal data. Knowledge verification is not consideredin this paper but implemented in AT-TECHNOLOGY - as well.
We studied the distributed knowledge acquisition processwhere temporal databases in medical domain were used asadditional sources of knowledge. We noticed that knowledgebase growth ratio lies between 12-25 % with 15 % in average.[4], [9]
IV. SOME ASPECTS OF THE INTERACTIONS BETWEENTHE TEMPORAL SOLVER, AT-SOLVER AND THE
SIMULATION MODELING SUBSYSTEM
The important feature of the temporal solver is the closeinteraction with the all-purpose AT-SOLVER and the subsys-tem of the simulation modeling of the external environment(external world), which is an obligatory component of anydynamic IES. [2] The temporal solver, as well as the subsystemof simulation modeling, acts on the times and process of theinteraction between the temporal solver, while the subsystemof simulation modeling is carried out by data and commandexchange in the asynchronous mode.
Figure 1 [8] shows the chart of the interactions between thetemporal solver, all-purpose AT-SOLVER, and the simulationmodeling subsystem. These interactions are provided by jointfunctioning support facilities. Moreover, the components inter-act with the total working memory. It should be noted that theinteraction is carried out in two modes: the development of theapplied dynamic IESs (including the adjustment of a series ofIES prototypes) and functioning of the final prototype of thedynamic IESs. The first mode that is needed for dynamic IESconstruction is the first that was considered in the present paper.
The interaction between the components begins after thesupport facilities receive a message about the start of theadjustment of the developed prototype of the dynamic IES.At the initial stage, this is the configuration of components,including the setting of the duration of the operation cycle forthe simulation modeling subsystem, indicating the database forthe temporal solver and the AT-SOLVER.
The joint functioning support facilities provide the synchro-nization of component operation by sending messages withstart or stop commands. The selected objects, whose collectionof attributes describes the system state, are presented in theworking memory. The knowledge base contains the temporalrules that are necessary to solve the formulated problem, aswell as to describe the events and intervals. As a result oftemporal inference on rules, the system state changes, i.e., theattributes of the working memory objects vary according to the
solved problem. A synchronous interaction would mean that thesimulation modeling subsystem goes to the standby mode up tothe inference completion by the temporal solver following datatransmission (from the subsystem to the solver). Otherwise, theasynchronous interaction is said to be the ability to continuethe operation of the subsystem without waiting for the temporalinference.
Asynchronous interaction allows higher productivity due tothe use of the time of the processing of general situationsby the temporal solver to execute the next modeling cycle.Note that similar asynchronous interactions are applied in realpractice, when it is impossible to react to an event immediately.This is the reason that the temporal solver and subsystem aresynchronous [8].
The functioning of the simulation modeling subsystem andthe inference facilities is an asynchronous process that isexecuted in parallel. The functioning of the temporal solverand AT-SOLVER is an synchronous process that is executedsequentially. The interaction between the components of theAT-TECHNOLOGY complex is a very difficult process thatrequires the development of models, methods, and softwarefacilities to support interactions. The functions of the modulesand blocks are the following: The configuration block carriesout the component configuration. It sets the duration of thecycle of the discrete model time and the assignment of thenames for the simulation modeling subsystem objects, temporalsolver, and AT-SOLVER.
The model time generation block counts the cycles of thediscrete model time according to the cycle duration specified bythe configuration block. The working memory scanning blockobserves the changes in the working memory.
The control effect calculation block implements the targetfunction of the interaction model. As a result of the blockoperation, the target component and the control effect, whichshould be set, are defined. The control effect generation blockforms the control effect as the message to the certain compo-nent. The interface module of the message exchange with thecomponents processes the input messages and sends controleffects. For joint functioning support facilities, the specialadjustment tools allow one to emulate the combined work ofthe components of AT-TECHNOLOGY in both the stepbystepand realtime modes. The use of these tools allows the study ofthe operation of the main components of the dynamic versionof AT-TECHNOLOGY in the fullest manner.
CONCLUSION
These experimental investigations showed the advantagesof the developed software tools compared to similar onesaccording to such criteria as the KRL power, operation speed,and reduction of the lead time of dynamic IESs. Verificationof the performance and efficiency of these tools was done bydeveloping a set of basic components, which is the minimumthat is needed for dynamic IES operation.
ACKNOWLEDGEMENTS
This work was partially funded by the RFBR (RussianFoundation for Basic Research), project No. 18-01-00457.
REFERENCES
[1] G. V. Rybina, Theory and technology of construction of integratedexpert systems. Monography. Moscow: Nauchtehlitizdat, 2008.
[2] G. Rybina, Intelligent systems from A to Z. A series of mono-graphs in three volumes. Vol. 2. Intelligent interactive systems.Dynamic Intelligent Systems. Moscow: Nauchtehlitizdat, 2015.
[3] G2 platform. [Online]. Available: http://www.gensym.com
131
Figure 1. Interaction diagram of temporal solver, all-purpose solver, and simulation modeling subsystem
[4] G. Rybina and I. Danyakin, “Combined method of automatedtemporal information acquisition for development of knowledgebases of intelligent systems,” Proceedings of the 2017 2nd Inter-national Conference on Knowledge Engineering and Application.London IEEE, pp. 117–123, 2017.
[5] J. Allen, “Maintaining knowledge about temporal intervals,”Communications of the Association for Computiong Machinery,pp. 832–843, 1983.
[6] G. Osipov, “Dynamic intelligent systems,” Iskusstvennyj intellekti prinyatie reshenij, no. 1, pp. 47–54, 2008.
[7] V. Emelyanov and Y. S.I., “Introduction to intelligent simulationof complex discrete systems and processes. language rbd, anvik,”1998.
[8] G. Rybina and A. Mozgachev, “The use of temporal inferencesin dynamic integrated expert systems.” Scientific and TechnicalInformation Processing, no. 6, pp. 390–399, 2014.
[9] G. Rybina and D. V. Demidov, “Automated acquisition, rep-resentation and processing of temporal knowledge in dynamicintegrated expert systems. postproceedings of the 9th annualinternational conference on biologically inspired cognitive archi-tectures, bica 2018 (ninth annual meeting of the bica society),vol.145, 2018. p. 448-452.”
[10] I. Efimenko, “Semantic of time: models, methods, and identi-fication algoritms for nlp-systems,” Vestnic Moskovskogo gosu-darstvennogo oblastnogo universiteta Linguistics, pp. 179–185,2007.
[11] N. Arutyunova and Y. T. E., “Logical analysis of language:Language and time,” lndrik, 1997.
[12] A. Tzacheva, A. Bagavathi, and P. Ganesan, “Mr - randomforest algoritm for distributed action rules discovery,” Interna-tional Journal of Data Mining & Knowledge Managment Process(IJDKP), no. 6(5), pp. 15–30, 2016.
[13] M. Kaufmann, A. Manjili, P. Vagenas, P. Fisher, D. Krossmann,F. Faerber, and N. May, “Timetable index: A unified data structurefor processing queries on temporal data in sap hana,” Proceedingsof the 2013 ACM SIGMOD International Conference on Manage-ment of Data, pp. 1173–1184.
ДИНАМИЧЕСКИЕ ИНТЕГРИРОВАННЫЕЭКСПЕРТНЫЕ СИСТЕМЫ : ОСОБЕННОСТИАВТОМАТИЗИРОВАННОГО ПОСТРОЕНИЯТЕМПОРАЛЬНЫХ БАЗ ЗНАНИЙ НА ОСНОВЕ
ЗАДАЧНО-ОРИЕНТИРОВАННОЙМЕТОДОЛОГИИ
Рыбина Г.В., Сорокин И.А., Сорокин Д.О.
Развитие инженерии знаний привело к появлению новойпрофессии, в которой активно востребованы как професси-ональные компетенции, так и индивидуальные качества лич-ности. Проанализирован методический и технологическийопыт автоматизированного построения компетентностно-ориентированных моделей специалистов в области инже-нерии знаний, в частности, специалистов по профессии"системный аналитик с использованием обучающих интегри-рованных экспертных систем.
Received 10.01.19
132
Hybrid intelligent multiagent model ofheterogeneous thinking for solving the problem
of restoring the distribution power grid afterfailures
Alexander KolesnikovImmanuel Kant Baltic Federal University
Kaliningrad Branch of the Federal Research Center“Computer Science and Control“
of the Russian Academy of SciencesKaliningrad, Russia
Sergey ListopadKaliningrad Branch of the Federal Research Center
“Computer Science and Control“of the Russian Academy of Sciences
Kaliningrad, [email protected]
Abstract—Problems arising in such dynamic environments asregional power grid have the following features: partial observ-ability, high dimensionality of the state space, interconnection anddependence of solutions on each other, which do not allow correct-ing the erroneous solution in the future. Abstract-mathematicalmodels are limited and irrelevant to such dynamic environments.For this reason teams of experts of various specialties or theircomputer models are involved, but even they do not alwayssuccessfully solve emerging problems. The success of the teamdepends largely on the ability of the decision maker to organizethe process of heterogeneous collective thinking, the diagnosis ofcollective effects, the problems of group behavior and to choosecorresponding model of collective reasoning. Under the conditionsof time constraints in practice, it is not possible to organize such acomprehensive collective problem solving process. In this regard,the paper proposes the formalized model and the basic algorithmsof a new class of intelligent systems, namely hybrid intelligentsystems of heterogeneous thinking. Their main feature is modelingof the discussion management in the expert team by the facilitatorwith heterogeneous collective thinking methods. These methodswill allow the agent modeling facilitator’s actions to organizecommunication and diagnostics of collective effects, problems andadjustment of group behavior, ensuring the relevance of the systemto conditions of dynamic directly unobservable environments.
Keywords—heterogeneous thinking; expert team; hybrid intel-ligent multiagent system
I. INTRODUCTION
If there is a failure in the distribution power grid, therate of power supply restoration is critical [1], [2]. Inorder to reduce economic and social losses, the majorityof energy supplying organizations develop guidelinesand operational procedures for the restoration of powersupply. Such guidelines are created based among otheron the results of the analysis of previous accidentsby expert teams from power engineering organizations,representatives of design institutes who have developedthe generation and power grid complex of the organiza-tion, as well as representatives of manufacturers of theequipment being operated [3]. The guidelines regulatethe sequence of actions-steps of operational personnel
for the restoration of power grid modes. However, theemergency conditions of the power system may differsignificantly from those adopted during the developmentof the recovery plan, which reduces the likelihood ofsuccess of the actions, leading to unacceptable loads,voltage levels or protection systems [4]. Although, anyemployee can be called up to the appropriate controlroom by the request of the operating personnel andmust arrive immediately, it is not possible to organizea comprehensive collective solution to the problem dueto the limited time to make decisions.
In this regard, the development of intelligent auto-mated systems integrating the knowledge of experts ofvarious specialties, the coordination of several optimalitycriteria and the consideration of a multitude of restric-tions in the context of dynamically directly unobservableenvironments and lack of time to make decisions are rele-vant. In addition, in case of technological violations at thefacilities belonging to several operators or independentactions, the execution of which is assigned to substationpersonnel without prior dispatcher’s order or permission[5], it is important to ensure a common understanding ofthe current emergency situation and coordinated work.
To simulate such structures for information preparationand decision-making support it is proposed to combinethe hybrid intelligent approach of A.V. Kolesnikov [6],the apparatus of multiagent systems in the sense of V.B.Tarasov [7] and the methods of heterogeneous thinking[8]–[10]. The result should be a new class of intelligentsystems, namely the hybrid intelligent multiagent sys-tems of heterogeneous thinking (HIMSHT). The use ofHIMSHT for information preparation of decision-makingwill automate the activities of operational personnel forreceiving and processing information about the external
133
environment, the state of the management system, thecourse of the controlled process, its analysis, modelingof the emergency situation and developing options toeliminate it by simulating collective problem solvingusing heterogeneous thinking methods. The result ofsuch work is new images, visual forms with a definitesemantic load [11], allowing the dispatcher team to seethe problem as a whole, its solutions, forecast of thesituation development in each case, to adopt an actionplan of eliminating the failure and to coordinate actionsin its implementation.
II. POST-EMERGENCY POWER GRID SUPPLYRESTORATION PROBLEM
Emergency is the state of the power grid (PG), associ-ated with changes in the normal operation of equipment,creating the risk of an accident [12]. Emergency sites ofthe PG should be shut down within milliseconds, and thesystems are divided into subsystems unbalanced by loadand generation a few seconds later. Supplier shutdownsoccur only a few minutes after division, and systems arerestored a few hours or days after redemption [13]. PGrestoring process is to build up its structure through timecoordinating the preparation and input of many inter-dependent objects that have retained functionality afteran emergency, as well as objects restored by personnelactions [14].
Planning for the restoration of the power system isa combinatorial problem that requires extensive knowl-edge, includes many constraints and conditions, whichoperator’s estimates are necessary further complicatingits integrated solution [1]. Three main features attributethis problem to especially interesting for modern plan-ners: partial observability, dimension of the state space,which makes a complete enumeration of states absolutelyimpossible, the consequences of the actions are difficultto simulate [15]. There are many statements of thisproblem, and new recovery methods are proposed thatare alternative to the commonly used procedures. Mostof them consider the problem in a simplified, “game”form, because of which they stop at the stage of de-velopment of a prototype [1]. Such “game” power gridsupply restoration planning (GSRP) can be representedas follows.
At GSRP, the power grid is represented by a graphPS =< V,E > with three types of nodes V : the powercenter (supplier) vs ∈ V s ⊆ V , the consumer (load)vl ∈ V l ⊆ V , and the bus vb ∈ V b ⊆ V . The edgesE of the graph denote electrical power lines withswitches opening or closing the line. Powered linesform a radial structure, i.e. there are no cycles ofpower lines. The power center is characterized by themaximum generated power, the consumer is character-ized by the nominal power consumption and the state(powered/disconnected), and the power transmission line
is characterized by the carrying capacity (maximumtransmittable power), the state (on/off) and operability(good/accident). It is required to determine which linesneed to be turned on/off, and in what order, to ensurethe maximum possible amount of power consumptionwhile observing the following operational limitations:maintaining the radial structure of the powered lines; foreach line, the total value of loads that are fed from thepower center through it should not exceed its carryingcapacity; consumers not affected by the initial shutdownshould not be turned off as a result of switching.
An example of the grid in GSRP is shown in Fig. 1.As seen in the left part of the figure in the normalstate, all the loads are distributed between the two powercenters, and there are no rings of switched on lines. Whenan accident occurs in such network, three consumers,indicated by dashed arrows, are de-energized. On thebottom of the Fig. 1 a situation is shown when all threeconsumers cannot be powered through one feeder dueto an overload. In this case, the out of service part ofthe power grid is divided into two parts by opening thenormally closed switch. After that, it becomes possibleto power de-energized consumers from different feedersof the functioning part of the network.
GSRP can be used to test optimization methods forthe purpose of their subsequent coarse-grained or fine-grained hybridization to solve the real problem of powergrid supply restoration planning (RSRP). Solutions ob-tained by GSRP problem solving methods without theirhybridization are irrelevant to RSRP because of thesignificantly larger number of object types and theirproperties that must be taken into account for construct-ing an acceptable plan in the latter, as well as non-factors in the sense of A.S. Narinyani [16] inherent inRSRP. The need to increase the number of types ofobjects being modeled is associated, for example, withthe impossibility of remote switching in some parts ofthe power grid, the presence of distributed generationand active consumers, and the need to take power gridphysical processes into account. The number of Non-factors of RSRP include, for example, the following:the underdetermined nature of the accident site duringrecovery planning; the inaccuracy of the amount of powerconsumed by each client and the distributedly generatedpower by each source; the fuzziness of the restorationtime of the power grid elements; incorrect operation ofemergency mode sensors; incomplete power grid model.
Based on the analysis of the papers [1], [15], [17]–[27] devoted to the post-emergency power grid supplyrestoration, the model of RSRP represented by the tuplewas developed:
RSRP =< PGE,PGR,PL,RT,RC, V H,RS,ACT >,
where PGE is the elements of the power grid; PGR isthe set of incident relations between the elements of the
134
Figure 1. Examples of graphs in the “game” planning of the supplyrestoration
power grid; PL is the set of locations; RT is the set ofroutes between locations; RC is the set of repair crews;V H is the set of vehicles; RS is the set of resources torestore the power supply; ACT is the set of actions torestore the power supply. The following elements of thepower grid are distinguished:
PGE =< PGEps, PGEco, PGEbu, PGEsw, PGEpl >,
where PGEps is the power source of distributedgeneration, PGEco is the consumer, PGEbu is thebus, PGEsw is the switch, PGEpl is the powerline. The power source of distributed generation ischaracterized by the following properties: operability(healthy/accident), state (disconnected/connected), his-tory of generation of active power, history of generationof reactive power, nominal voltage, parameters of thetransition process at cold start, location. The consumeris characterized by the following properties (it is assumedthat consumers can generate and deliver excess electric-ity to the grid): state (disconnected/connected), priority,history of generation and consumption of active power,history of generation and consumption of reactive power,nominal voltage, cold start transient parameters, location.
A bus, a switch and a power line are characterized bythe following general properties: operability, long-termallowable current, allowable transmitting active power,allowable transmitting reactive power, coefficient of al-lowable overload for a given time, location. A switchis additionally characterized by the following properties:switch state (on, off, or disabled with no turn on), switchtype (remote/local), synchronization ability. Power linehas properties: voltage loss, active power loss, reactivepower loss.
Location pl ∈ PL is the geographical coordinatesand/or address of power grid element of set PGE,car depot or resource warehouse. The route rt ∈ RTis described by the initial location, final location, traveltime, the expected delay in travel in the form of sta-tistical or fuzzy variable. The repair crew rc ∈ RC ischaracterized by the following properties: the number ofemployees, the level of admission. The vehicle vh ∈ V Hhas the following properties: the depot location, themaximum number of passengers, carrying capacity, thedimensions of the cargo compartment. Properties ofresource rs ∈ RS for the restoration of the power gridare the weight, the dimensions, and the location. Theaction act ∈ ACT on the restoration of the power gridis characterized by the following properties: the objectof restoration, the duration, the expected repair delay,described by statistical or fuzzy variables, the level ofpersonnel admission, the necessary resources.
It is required to make a power grid restoration plan,which includes the sequence of turning on and off theswitches, the sequence of trips of repair crews to performswitching and repair work.
Criteria for optimality of the plan are following: min-imizing the shutdown time of the priority consumers,maximizing the total recovered load, maximizing thereliability of the power system (the stability of the powersystem to subsequent accidents).
The following restrictions apply to the plan: the preser-vation of the radial structure of the network of poweredlines; for each line, the total value of loads that aresupplied from a source of distributed generation throughit should not exceed its carrying capacity; active powerbalance must be maintained; reactive power balance mustbe maintained; voltage and frequency values must bewithin acceptable limits; consumers not affected by theinitial outage should not be turned off as a result of theswitch; the repair actions must be carried out by teamsthat have the appropriate admission if the necessary re-sources are available in their vehicle; vehicle capacity isnever exceeded; brigade working time is limited; vehiclesmust return to their depot; when forcibly dividing the gridto islands, the communication lines between the islandsmust have synchronization equipment for the subsequentmerge of the islands.
To solve the RSRP, it is proposed to model the135
Figure 2. Rhombus of group decision making by S. Keiner, K. Toldi,S. Fisk, D. Berger
collective decision-making by the operating personnelof the energy supplying organization, power engineers,logisticians, and labor protection specialists with theHIMSHT.
III. TEAM DECISION MAKING IN POWER GRIDMANAGEMENT
When solving new, previously not encountered prob-lems, teamwork in general case consists of the fol-lowing stages: formulation, analysis of the problem,data collection and interpretation, search for solutions,analysis of the effectiveness of solutions and final choice,presentation of results, implementation of the solution,monitoring and evaluation of results [28]. The problemsolving process is superimposed on the process of form-ing and developing the team as a single entity. The latterconsist of the stages: formation, turbulence, refinementof proposals and preparation of alternatives, decisionmaking and disbandment [29], [30], which is consistentwith team decision making model by S. Keiner, K. Toldi,S. Fisk, D. Berger (Fig. 2) [9].
At the first stage, members of the team get to knoweach other, exchange official information about eachother, make suggestions on the teamwork, adhere togenerally accepted points of view, and propose obvioussolutions [30]. If the problem has an obvious solution,the discussion ends, otherwise the divergent thinkingprocess begins, within which a non-judgmental discus-sion and the generation of a large number of solutionsare encouraged [9]. If the team managed to go beyondthe boundaries of traditional opinions, the process ofdiscussion goes into the turbulence stage, when conflictscould arise between team members due to conflictingsolutions. By conflict we will understand the situation ofthe disagreement of two or more experts about knowl-edge, belief, opinions, i.e. cognitive conflict [31]. Theconflict is a distinctive feature of the turbulence stage,which allows the facilitator to take measures to developmutual understanding and bring together the experts’points of view.
At the stage of finalizing proposals and preparingalternatives, experts formulate specific proposals from
valuable ideas and “grind” them until all the discussionparticipants come to a final solution embodying all thediversity of points of view. This stage is characterizedby “convergent thinking”, i.e. the classification of ideas,their generalization, and making assessments. Duringdecision-making and disbandment stage the team inte-grated problem solution is developed, taking into accountthe opinions of all the participants in the discussion.
The rhombus of group decision-making model canbe used by the facilitator or his model to identify thecurrent situation of decision-making and to attempt tosteer the discussion in the required direction, activatingthe appropriate thinking style in the team.
IV. FORMAL MODEL OF THE HYBRID INTELLIGENTMULTIAGENT SYSTEM OF HETEROGENEOUS THINKING
IN COLLECTIVE OPERATIONAL WORK
Formally HIMSHT is defined as follows:
himsht =< AG∗, env, INT ∗, ORG, ht >,
acthimsht =
( ⋃ag∈AG∗
actag
)∪
∪actdmsa ∪ acthtmc ∪ actcol,actag =< METag, ITag >, ag ∈ AG∗,∣∣∣∣
⋃ag∈AG∗
ITag
∣∣∣∣ > 2,
where AG∗ = ag1, ..., agn, agdm, agfc is the setof agents, including expert agents (EA) agi, i ∈ N,1 6 i 6 n, decision-making agent (DMA) agdm, andfacilitator agent (FA) agfc; n is the number of EA; envis the conceptual model of the external environment ofHIMSHT; INT ∗ = prot, lang, ont, dmscl are theelements for structuring of agent interactions: prot is theinteraction protocol; lang is the message language; ontis the domain model; dmscl is the classifier of collectivesolving problem situations, identifying the stages of thisprocess (Fig. 2); ORG is the set of HIMSHT architec-tures; ht is the set of conceptual models of macro-level processes in the HIMSHT: ht is the model of thecollective problem solving process with heterogeneousthinking methods (Fig. 2); acthimsht is the functionof the HIMSHT as a whole; actag is the function ofEA from the set AG∗; actdmsa is the FA’s function“analysis of the collective problem solving situation”;acthtmc is FA’s function “choice of heterogeneous think-ing method”; actcol =< metma, itma > is the collec-tive dynamically constructed function of HIMSHT withmultiagent method metma and intelligent technologyitma; metag ∈METag is the problem solving method;itag ∈ ITag is the intelligent technology, with whichthe method metag is implemented; “∪” is the unionoperation over tuples or sets.
To implement the FA function “analysis of the col-lective problem solving situation”, the concepts of com-patibility of the partial solutions proposed by EA, the
136
intensity of the conflict between them, and the stageof the problem solving process are introduced. Solutioncompatibility cmp is problem-depended scalar functiondescribing the possibility of simultaneous implementa-tion of two partial solutions.
The intensity of the conflict between the two agentsis based on the compatibility of the partial problemsolutions offered by them:
cnf(agi, agj) =Ni∑k=1
Nj∑l=1
cmp(deck, decl)(NiNj)−1,(1)
rres res1 (deck, agi) rres res
1 (decl, agj),
where Ni, Nj are the number of private solutions foundby agents agi and agj , respectively; rres res
1 is therelation “to be found by” between the private solutionand the agent who proposed it; “” is the operation ofgluing concepts.
The conflict intensity in HIMSHT as a whole de-scribed by the expression
cnfhimsht =
n∑
i=1
n∑
j=i+1
2cnf(agi, agj)(n− 2)!(n!)−1, (2)
where “!” is factorial.The conflict intensity between agents or in HIMSHT
as a whole is used as a universe of the linguistic variable“conflict”, which is then used in the implementation ofthe function “choice of heterogeneous thinking method”acthtmc. The linguistic variable “conflict” is representedby the expression
cnfl =< β, T, cnf,G,M >, (3)
where β = “conflict” is the name of the linguistic vari-able; T = “absent”, “minor”, “moderate”, “sharp”is the term-set of its values, i.e. the names of fuzzyvariables; cnf = [0, 1] is the universe of fuzzy variables;G = ∅ is the procedure for the formation of new termsusing the elements of the set T ; M = µabsent(cnf),µminor(cnf), µmoderate(cnf), µsharp(cnf) is the pro-cedure that assigns meaningful content to each term ofT by composing a fuzzy set.
The value of the character variable “stage of theproblem solving process” stg, defined on the setSTG = “divergent”, “turbulence”, “convergent”,is calculated according to the rules:
stg = “divergent” ∧ (cnfl = “moderate” ∨∨cnfl = “sharp”) −→ stg = “turbulence”, (4)stg = “turbulence” ∧ (cnfl = “absent” ∨∨cnfl = “minor”) −→ stg = “convergent”. (5)
The algorithm of the FA’s function “analysis of thecollective problem solving situation” is the sequence ofsteps:
1) set the initial values: stg = “divergent”,cnf(agi, agj) = 0, cnfhimsht = 0, cnfl = “absent”;
2) expect messages;3) if a message is received about HIMSHT termination,
then the end;4) if a message is received about the solution deck devel-
oped by the EA agi, then proceed to step 5, otherwisethe abort execution with error;
5) determine cnf(agi, agj) for each EA agj by (1), wherej ∈ N, 1 6 j 6 n, j 6= i;
6) calculate cnfhimsht and cnfl by (2) and (3), respec-tively;
7) determine the stage stg of the problem solving processaccording to the rules (4) and (5);
8) go to the step 2.
The FA’s function “choice of heterogeneous thinkingmethod” is implemented using a fuzzy knowledge baseabout the effectiveness of heterogeneous thinking meth-ods depending on the characteristics of the problem,the stage of its solution and the current solution situ-ation in HIMSHT. This fuzzy knowledge base shouldbe developed based on the results of the computationalexperiments to be carried out with algorithms that imple-ment these methods. By now the comparative analysis ofthe approaches proposed in HIMSHT and implementedin hybrid intelligent multi-agent systems, that increasedefficiency by more than 7% solving complex transport-logistic problem [32], suggests the advantages of theformer and their greater relevance to the problems indynamic environments.
Thus, thanks to the FA, which initiates the use ofvarious methods of heterogeneous thinking, and EAs,which implement various technologies of artificial intel-ligence, the HIMSHT dynamically rebuilds the algorithmof its functioning, each time developing a hybrid intel-ligent solution method that is relevant to the dynamicproblem. HIMSHT combines the representation of theheterogeneous functional structure of a problem withheterogeneous collective thinking of intelligent agents,which creates conditions for solving the problem withoutsimplifying in the dynamic environment of the regionalpower grids.
V. CONCLUSION
In this paper, the main stages of solving problems by theteam of experts engaged in operational activities are reviewed,and the thinking styles of participants are highlighted in thesestages. The formalized model of the HIMSHT and algorithmsthat simulate heterogenous thinking processes are proposedfor the relevant modeling of the problem solving process ina small team of experts. The proposed HIMSHT moves animitation of collective development of operational actions tothe field of synergetic informatics, when interaction of agentsis necessary for obtaining a result greater than the sum of thework carried out individually. This leads to self-organizing,social management models, each element of which is develop-ing, obtaining data and knowledge from other elements. Thisreduces the cost of developing and operating the system. Theuse of such systems in operational dispatching (operational andtechnological) management will make it possible to developsolutions relevant to the problems that arise in the complex,dynamic environments of regional power grids.
137
ACKNOWLEDGMENT
The reported study was funded by RFBR according to theresearch project No. 18-07-00448A.
REFERENCES
[1] M.I. Uspenskiy, and I.V. Kyzrodev, Metody vosstanovleniya elektro-snabzheniya v raspredelitel’nykh setyakh [Methods for restoring electricalsupply in distribution networks], Syktyvkar, 2010. (in Russian)
[2] Avtomatizatsiya dispetcherskogo upravleniya v elektroenergetike [Automa-tion of dispatching control in electric power industry], Yu.N. Rudenko,and V.A. Semenova (eds.), Moscow: Izdatel’stvo MEI, 2000. (in Russian)
[3] Avariya v energosisteme Kaliningradskoy oblasti [Accident inthe power system of the Kaliningrad region]. Available at:http://allabc.ru/energetika/2212-avariya-v-energosisteme-kaliningradskoj-oblasti.html. (accessed 25.12.2018) (in Russian)
[4] A.L. Barabashi, and E. Bonabo, “Bezmasshtabnyye seti [Non-scale net-works ]" in V mire nauki [In the world of science], 2003, No 8, pp.54-63.
[5] A.A. Filatov, Obsluzhivaniye elektricheskikh podstantsiy operativnympersonalom [Maintenance of electrical substations operational staff ],Moscow: Energoatomizdat, 1990. (in Russian)
[6] A.V. Kolesnikov, Gibridnye intellektual’nye sistemy. Teoriya i tekhnologiyarazrabotki [Hybrid intelligent systems: theory and technology of develop-ment], Saint Petersburg: Izdatel’stvo SPbGTU, 2001. (in Russian)
[7] V.B. Tarasov, Ot mnogoagentnykh sistem k intellektualnym organizatsiyam:filosofiya, psikhologiya, informatika [From multiagent systems to intelligentorganizations: philosophy, psychology, and informatics]. Moscow: Edito-rial URSS, 2002. (in Russian)
[8] H. Gardner, Multiple Intelligences – The Theory in Practice. New York:Basic Books, 1993.
[9] S. Kaner, L. Lind, C. Toldi, S. Fisk, and D. Beger, The Facilitator’s Guideto Participatory Decision-Making, San Francisco, CA: Jossey-Bass, 2011.
[10] E. de Bono, Parallel Thinking: From Socratic to De Bono Thinking,Melbourne: Penguin Books, 1994.
[11] B.A. Dushkov, A.V. Korolov, and B.A. Smirnov, Osnovy inzhenernoypsikhologii [Fundamentals of engineering psychology]. Moscow: Aka-demicheskiy prospect, 2002. (in Russian)
[12] V.V. Krasnik, Potrebiteli elektricheskoy energii, energosnabzhayushchiyeorganizatsii i organy Rostekhnadzora. Pravovyye osnovy vzaimootnosheniy[Consumers of electrical energy, energy supplying organizations andbodies of Rostechnadzor. Legal basis of relationships], Moscow: NTSENAS, 2005. (in Russian)
[13] M.M. Adibi, and L.H. Fink, “Overcoming Restoration Challenges As-sociated with Major Power System Disturbances" in Power and EnergyMagazine, IEEE, 2006, Vol. 4(5), pp. 68-77.
[14] O.A. Masterova, and A.V. Barskaya, Ekspluatatsiya elektroenergetich-eskikh sistem i setey [Operation of power systems and networks], Tomsk:TPU, 2006. (in Russian)
[15] S. Thiebaux, and M.-O. Cordier, “Supply Restoration in Power DistributionSystems — A Benchmark for Planning under Uncertainty" in Proceedingsof the 6th European Conference on Planning (ECP-01), Toledo, 2001, pp.85-96.
[16] A.S. Narinyani, “Inzheneriya znaniy i NE-faktory: kratkiy obzor-08[Knowledge Engineering and Non-Factors: A Brief Overview-08]" inVoprosy iskusstvennogo intellekta [Artificial Intelligence Issues], 2008, No,pp.61–77. (in Russian)
[17] I.V. Kyzrodev, and M.I. Uspensky, “Avtomatizatsiya vosstanovleniya elek-trosnabzheniya v raspredelitel’nykh setyakh [Automating the restorationof power supply in distribution networks]" in Izvestiya Komi nauchnogotsentra UrO RAN [Izvestiya Komi Scientific Center, Ural Branch of theRussian Academy of Sciences], 2010, No 2, pp. 84-91. (in Russian)
[18] V. Kumar, H.C.R. Kumar, I. Gupta, and H.O. Gupta, “DG integratedapproach for service restoration under cold load pickup" in IEEE Trans-actions on Power Delivery, 2010, Vol. 25(1), pp 398-406.
[19] P. Hentenryck, and C. Coffrin, “Transmission system repair and restora-tion" in Mathematical Programming. Ser. B, 2015, Vol. 151, pp. 347–373.
[20] W. Chen, M. Tsai, and H. Kuo, “Distribution system restoration using thehybrid fuzzy-grey method” in IEEE Transactions on Power Systems, Vol.20(1), pp. 199–255.
[21] S. Jayasinghe, and K. Hemapala, “Multi agent based power distributionsystem restoration – a literature survey” in Energy and Power Engineering,2015, Vol. 7, pp. 557-569.
[22] J.W. Feltes et al., “Some considerations in the development of restorationplans for electric utilities serving large metropolitan areas” in IEEETransactions on Power Systems, 2006, Vol. 21, pp. 909-915.
[23] M. Gholami, J. Moshtagh, and N. Ghadernejad, “Service restoration in dis-tribution networks using combination of two heuristic methods consideringload shedding” in Journal of Modern Power Systems and Clean Energy,2015, Vol 3(4), pp. 556–564
[24] P. Gomes, A.C.S. de Lima, and A. de P. Guarini, “Guidelines for PowerSystem Restoration in the Brazilian System” in IEEE Transactions onPower Systems, 2004, Vol. 19(2), pp. 1159 – 1164.
[25] S.-J. Lee, S.-I.Lim, B.-S. Ahn, “Service Restoration of Primary Distri-bution Systems Based on Fuzzy Evaluation of Multi-Criteria“ in IEEETransactions on Power Systems, 1998, Vol. 13(3), pp. 1156 – 1163.
[26] Y. Besanger, M. Eremia, and N. Voropai, “Major grid blackouts: Analysis,classification, and prevention” in Handbook of Electrical Power SystemDynamics: Modeling, Stability, and Control, New Jersey: Wiley – IEEEPress, 2013, pp. 789-863.
[27] N.I. Voropay, and Buy Din’ Tkhan’, “Vosstanovleniye sistemy elek-trosnabzheniya s raspredelennoy generatsiyey posle krupnoy avarii[Restoration of the power supply system with distributed generation after amajor accident ]” in Promyshlennaya energetika [Industrial Energy], 2011,No 8, pp.12-18. (in Russian)
[28] M.V. Samsonova, and V.V. Yefimov, Tekhnologiya i metody kollektivnogoresheniya problem: Uchebnoye posobiye [Technology and methods ofcollective problem solving: Textbook], Ul’yanovsk: UlGTU, 2003. (inRussian)
[29] A.N. Zankovskiy, Organizatsionnaya psikhologiya: Uchebnoye posobiyedlya vuzov po spetsial’nosti «Organizatsionnaya psikhologiya» [Organiza-tional psychology: Textbook for high schools on the specialty “Organiza-tional psychology"], Moscow: Flinta: MPSI, 2002. (in Russian)
[30] Organizatsionnoye povedeniye: Uchebnik dlya vuzov [Organizational Be-havior: A Textbook for Universities], G.R. Latfullin, and O.N. Gromova(eds.), Saint-Petersburg: ZAO Izdatel’skiy dom «Piter», 2004. (in Russian)
[31] A.Y.C. Tang, G.S. Basheer, “A Conflict Resolution Strategy SelectionMethod (ConfRSSM) in Multi-Agent Systems" in International Journalof Advanced Computer Science and Applications, 2017, Vol. 8(5), pp.398–404.
[32] A.V. Kolesnikov, I.A. Kirikov, S.V. Listopad, Gibridnye intellektual’nyesistemy s samoorganizatsiey: koordinatsiya, soglasovannost’, spor [Hybridintelligent systems with self-organization: coordination, consistency, dis-pute], Moscow: IPI RAN, 2014. (in Russian)
ГИБРИДНАЯ ИНТЕЛЛЕКТУАЛЬНАЯМНОГОАГЕНТНАЯМОДЕЛЬ ГЕТЕРОГЕННОГО
МЫШЛЕНИЯ ДЛЯ РЕШЕНИЯ ЗАДАЧИВОССТАНОВЛЕНИЯ РАСПРЕДЕЛИТЕЛЬНОЙ
ЭЛЕКТРОСЕТИ ПОСЛЕ АВАРИЙ
Колесников А.В., Листопад С.В.
Проблемы, возникающие в таких динамических средахкак региональная распределительная электросеть, облада-ют следующими особенностями: частичная наблюдаемость,высокая размерность пространства состояний, взаимосвязьи зависимость решений друг на друга, не позволяющиеисправить ошибочное решение в будущем. Абстрактно-математические модели ограничены и нерелевантны такимдинамическим средам, в связи с чем привлекаются коллек-тивы экспертов различных специальностей или их компью-терные модели, но и они не всегда успешно решают возника-ющиепроблемы.Успех работыколлектива вомногом зависитот способности лица, принимающего решения, организоватьпроцесс гетерогенного коллективногомышления, диагности-ку коллективных эффектов, проблем группового поведенияи выбор на их основе соответствующеймодели коллективныхрассуждений. В условиях временных ограничений при реше-нии практических проблем организовать такое всестороннееколлективное решение проблемы не представляется возмож-ным. В этой связи в работе предлагается формализованнаямодель и основные алгоритмы нового класса интеллектуаль-ных систем – гибридные интеллектуальные системы гетеро-генного мышления. Особенность данного класса интеллек-туальных систем – моделирование действий фасилитаторапо управлению дискуссией команды экспертов методамигетерогенного коллективного мышления. Их применениепозволит агенту, моделирующему действия фасилитатора,организовать коммуникацию и диагностику коллективныхэффектов, проблем и корректировку группового поведения,обеспечив релевантность системы рассматриваемым пробле-мам в условиях динамических непосредственно ненаблюда-емых сред.
Received 09.01.19
138
Visual event-situational approach forinformation preparation of decisions andoperational technological management of
complex dynamic objects
Alexander KolesnikovInstitute of Physical and Mathematical Sciences and Information Technology,
Immanuel Kant Baltic Federal UniversityKaliningrad Branch of the Federal Research Center “Computer Science and Control“ of the Russian Academy of Sciences
Kaliningrad, [email protected]
Abstract—Operational work with the image of an objectnot directly perceived is a difficult task for the operator.The mapping methods used are not relevant to the mentalimage; they prevent the operator from performing actionsin the mind and contribute to data interpretation errors. Itis supposed to change the state of affairs using the visualevent-situational subject-oriented approach, which consistsin computer simulation of a multi-structural, multi-layeredby concreteness and clarity display. In this approach, thesubject action is considered as an ordered sequence oftwo-step transformations by the subject of a deformable,limited amount of mixed perceptual and subject-schematicinformation about the situations and state of the object,in the form of a reasonable, regulatory impact on it. Theresult of the implementation of this approach should be anew class of artificial intelligent systems, namely cognitivehybrid intelligent systems.
Keywords—visual event-situational approach, manage-ment of complex dynamic objects, cognitive hybrid intelli-gent systems
I. INTRODUCTION
Management in a dynamic environment is associatedwith heterogeneity and state of distribution in the spaceof the problems being solved or the systems beingdeveloped. The decisions are interconnected and im-pose restrictions on each other, making it impossibleto correct an erroneous decision in the future, sinceeven a small initial error rapidly increases over time.The processing of heterogeneous information through theintegration of expert knowledge has been investigatedin the researches on hybrid intelligent systems (HIS),synthesizing a method for solving a complex problemover a heterogeneous model field.
However, traditional HIS [1], [2] do not involvethe operator’s right-sided, visual-imaginative reasoning,complicating the operational work with an object thatis not directly perceived, forcing him to think logi-cally instead of providing intuitive decision-making. It
is proposed to overcome this drawback by obtainingnew knowledge about operational-technological, human-machine management of complex, dynamic systems,developing a visual event-situational subject-oriented ap-proach of information preparation of solutions and oper-ational technological management of complex dynamicobjects and its implementation in a new class of artificialintelligent systems, namely cognitive hybrid intelligentsystems. Visualization of spatial-temporal relations ofresources and actions for expressing the semantics ofthe state of a complex system, introduction of the psy-chologically justified heuristics of the functional systemof subject action and dynamic operational image in thesupporting solutions will return the subject of operationalactivities to the space of meanings, the space of things,properties and relations. It will promote the connectionof the sensual and the rational, the individual and theuniversal; make out the ideas emerging from him; showthe structure and dynamics of changes in the operationalimage; help to follow the movement of thought, fix it,facilitate mental activity, establishing cause-effect andfunctional relationships; significantly reduce the timeto solve operational tasks, reduce the risk of errors;qualitatively improve the system of artificial intelligence.
II. VISUAL MANAGEMENT AND CONTROL
It is believed that visual management was born in theconcept of Taichi Ono [3], who is the author of leanmanufacturing, the core of the Toyota system (Japan).Visual management is a clear, simple and effective wayto organize work and report on it, so that everyone cansee the work of everyone, and the organization becomes“transparent” [4], a process that provides the humanfactor with simple visual signals to immediately respondto new terms and conditions.
139
(a) (b)
(c)
Figure 1. Examples of visualization: (a) problem solving board in the traditionof “gemba” at GAZPROMNEFT-SNABZHENIYe LLC; (b) visualization at theoffice of the Atomic Energy State Corporation Rosatom; (c) an example of thevisualization of the entrance group in the workshop, signs of harmful substancesand noise exposure
One of them is gemba from Japanese managementpractice that is the controlling “walk” of decision makersto workplaces to inspect and evaluate what is happening.Visual control is an effective and self-regulating factorin the measurement of visual signals: plans, unfinishedtasks, inventory, resource consumption, and quality [5].The objectives of visualization of control are to seeproblems and understand situations in the workplace andto see the decisions, clarify actions to achieve goals. Themethods of visual control are key indicators, photos andmarkup (Fig. 1).
Another approach to the visual management has beenadopted in information, situational and dispatch centersthat are designed for deep immersion in the process,setting up constructive communications, prompt responseto key events and organizing a multi-level problem solv-ing system (Fig. 2) [6]. Morning, ten-minute “volatiles”are sufficient to obtain operational strategic information,which allows to see priorities, identify deviations, andoutline actions to correct the situation. A huge amountof data, previously scattered, is collected, systematizedand presented in a convenient for perception, digitizedform: graphs, charts, diagrams and tables.
Two approaches to centralized visual control areknown namely event and situation description of controlobjects in complex systems.
III. EVENT AND SITUATIONAL MANAGEMENT
L. Wittgenstein held an event-based view at the levelof logical, linguistic systems: “The world is a totality offacts, not objects ... Progressive, the fact is the existenceof events” [9]. Hence, the control object is considered asa stream of events through the set of which things are de-termined. If all the events in which the object participated
(a) (b)
Figure 2. Examples of visual management information centers: (a) complexsolution of the company POISK [7]: mnemonic scheme, plastic appliqué, LCDpanels; (b) S - 2000 dispatch board, built-in video cubes of NTK Interface LLC[8]
are recorded, then we will get its full description, not anabstract, but a concrete and comprehensive descriptionof how the object “looks” for a particular system, whatproperties, at what moments it has and how it participatesin the functioning of the system.
Event management is based on the event descriptionof complex systems, which is the process responsiblefor managing events during the life cycle, the mainactivity of operational management. Event is a changein the state, which is important for the management ofobjects and their relations. To be effective, operationalmanagement must know the state of the control objectand its elements, as well as track any deviations fromthe norm. Event management is a proactive and sys-tematic approach aimed at predicting problems, antic-ipating threats, minimizing surprises, making decisionson emerging issues. According to M. Lauzen, effectiveevent management requires two-way communication, aclear formal monitoring of the environment and active,meaningful strategies [10]. According to J. and V. Coates,event management is an organized activity to identifyemerging trends, troublesome or controversial issues thatmay affect the organization over the next few years, andto develop a wider and more positive response range oforganizations in relation to the future [11]. The orderof event management is following: detection, filtering,prioritization, correlation, determination of the responsemethod.
The entity approach, entity thinking is traditional forthe modern man, for whom the description of the worldas a multitude of spatially localized objects-entities-resources is peculiar. Things are predetermined. The rela-tionship of objects is described through relations. Modernmethods of describing or modeling complex systemsadhere to the entity ontology: at first decompositionand selection of objects, then their classification, withattributing properties to objects and establishing relationsbetween them (“part-whole”, “gender-type”, “depends”,etc.). A.I. Uyomov [12] held an entity-based view at thelevel of logical, linguistic systems, who expressed ownworld view through the “entity-property-relation" triad,which was adapted by A.V. Kolesnikov, I.A. Kirikov and
140
V.F. Ponomarev [13] to the “resource-property-action"triad in relation to the operating activities of dispatchersin complex systems.
Situational approach is based on the entity thinking.In the 20s of the last century, M. Follett spoke about the“law of the situation”. She noted that “different situa-tions require different types of knowledge” and differentresponses. A fully situational approach was developed inthe late 60s. The entity approach to the ontology of theworld was reflected in the situational approach, it wasformed in the USA in the late 1960s. Representativesare F. Kast [14], H. Koontz [15], P. Lawrence, J. Lorsch[16], J. Thompson [17], etc. The term “situation manage-ment” was formed in the 60s of the last century, whenSoviet scientists Yu.I. Klykov and D.A. Pospelov [18]introduced this concept. It appeared due to the develop-ment and application of logical-linguistic models to themanagement practice, when situations were described inthe language of qualitative concepts and relations, andthe means of mathematical logic were used to organizesymbolic transformations from the initial situation to thetarget one.
To develop a visual event-situational subject-orientedapproach, it is proposed to develop the concept of rela-tional, symbolic-logical language for describing the stateand situations of the control object proposed by A.V.Kolesnikov and V.F. Ponomarev [1] on the basis of thesituational approach and the entity view of A.I. Uymov[12].
IV. CONSTRUCTING OF VISUAL-SYMBOLICSTATEMENTS ABOUT THE POSITION OF RESOURCES
(RESOURCE-BASED SITUATION, R-SITUATION)
Definition 1. R-situation (resource-based situation) isa set of spatial relations on resources used in productionoperations at a given point in time and in the context ofthe spatial structure of the control object.
Position is a characteristic of the spatial relationshipof one of the object’s elements to other elements withina certain area. Graphically the position is usually shownin a diagram of related elements, for example, in a streetand neighborhood map. The position is interpreted as“resource-based situation” or “r-situation”. In Fig. 3(a)graphic statements about the position of resources (aboutthe r-situations) are shown by means of the graphicinterface of the online service “Bus Time” in Kaliningrad(www.bustime.ru).
Monitoring of weather conditions (weather r-situation)that are graphical statements about the weather r-situationcan be performed by pictograms (Fig. 3(b)) accordingto data widely presented by meteosites on the Internetin two aspects: current weather conditions and forecastof weather conditions on the required number days.The symbolization and the schematization of resource-based situations as applied to sectioned power grids
(a)
(b)
Figure 3. Graphic statements: (a) about the situation (r-situation) in the onlineservice “Bus Time” (www.bustime.ru) in Kaliningrad; (b) icons for monitoringweather conditions (left - right, top - down): cloudy, rain, temperature, partlycloudy, wind direction, wind force, freezing rain, clear, thunderstorm, tornado,snow
should visually express the correlation of variabilityand constancy, which requires displaying the positionas a characteristic of the spatial relation of one of theresources (dynamic) to other resources (static) withinsome area (also static resource) at some point in time.So, we interpret three components of the p-situation: 1)a dynamic resource; 2) a group of static resources; 3) anarea of establishing spatial relations “resource-resource"between the first and second resources. The dynamicresource is electricity, characterized by three main pa-rameters: voltage (volts), amperage (amperes) and power(watts). It is the object of the operation “transmission”over a distance within the “resource-action” relation.Static resources are the power grids, their devices andinstallations, i.e. the means of electricity transmissionthat are characterized by the main parameter - trans-mission capacity. They are the means of transmissionof electric energy over a distance within “resource-action" relation. In general case, a power transmissionline should be considered as an object with parametersdistributed along one spatial coordinate (along a line).One of the main indicators of power transmission isthe transmission power, i.e. the amount of energy trans-mitted per unit of time. The transmission capacity isthe highest power that, given all technical limitations,could be transmitted through the line. For example, fora 110 kV line, the capacity is 30 MW. Power grids arecharacterized by a number of indicators, which primarilyinclude the magnitude of the transmitted power, nominal
141
(a) (b)
(c) (d) (e) (f)
Figure 4. Graphic statements: (a) about the resource “section of the overheadpower line”; (b) about the resource “electricity"; (c) about the role-based visualrelation “resource-action”, where “overhead power line section” resource and“transfer electricity” action are simplified; (d) “section of the overhead powerline is a mean of electricity transmission"; (e) “section of the overhead powerline, as a mean of electricity transmission, is under load (powered)"; (f) “sectionof the overhead power line, as a mean of electricity transmission, is de-energized”
voltage, functional value and transmission distance, theconfiguration (topology) of the network. The area ofspatial relations “resource-resource" is a region, i.e. apart of the space occupied by the resource in its naturalenvironment. This could be the physical place shown inconnection with the natural characteristics of the placeitself and its immediate surroundings. This could bethe formalized place, showing the structural nature ofa place in terms of its essential characteristics withthe replacement of natural properties by a simplifiedform, which does not reproduce the exact image of theresource, but gives its generalization. This is the casewhen technical clarity is more important than visuallyperceived reality. Fig. 4 depicts a graphic statement aboutthe resource “overhead power line section”.
The construction of complex graphic statements “over-head power line section as a mean of power transmissionthat contactly connects transformer and recloser is pow-ered” and “two-sections branch of the overhead powerline is powered” is shown in Fig. 5.
The complex graphic statement about the p-situation“normal power transmission mode, the emergency trans-fer switch resource is off" is shown in Fig. 6(a). Fig. 6(b)shows the role visual relation “resource-resource”. Therole on the left is occupied by the complex graphic state-ment about the resource “emergency power transmissionmode, the emergency transfer switch resource is on”.
Fig. 7 depicts graphic statement about r-situation “nor-mal power transmission mode” and its position.
V. CONSTRUCTING VISUAL-SYMBOLIC STATEMENTSABOUT ACTIONS THAT ARE SIMULTANEOUSLYPERFORMED (OPERATION-BASED SITUATION,
O-SITUATION)
Definition 2. O-situation (operation-based situation) is“simultaneously” relation over the set of operations with
(a) (b)
(c)
(d) (e)
(f) (g) (h)
Figure 5. Graphic statements: (a) about the resource “transformer”; (b) aboutthe resource “substation”; (c) about the role visual relation “resource – resource”(“contact connection”); (d) “contact connection” is simplified; (e) two role visualrelations “resource – resource” form the composite graphic statement “overheadpower line section that contactly connects transformer and recloser is powered”;(f) about role visual relation “resource-resource" (“branch", “tap off"); (g) “two-sections branch of the overhead power line is powered”; (h) “two-sections branchof the overhead power line, intended for electricity transmission, is powered”
(a)
(b)
Figure 6. Graphic statements about r-situation: (a) “normal power transmissionmode, the emergency transfer switch resource is off"; (b) “emergency powertransmission mode, the emergency transfer switch resource is on”
Figure 7. Graphic statement about r-situation “normal power transmissionmode” and its position (shown schematically)
142
(a) (b)
(c)
Figure 8. Complex graphic statements: (a) about the role visual relation “action- action” (“simultaneously”), the symbols of actions are not detailed; (b) about theparameters, essential for the problem solved at the moment t, and the qualitativefuzzy characteristics of the resources (role visual “property - resource” relations);(c) about the state of the visual control object at time
the resources of the control object at a given time inthe context of its production structure (process). Sincethe operation scenario is a plan that forms the current r-situation, the o-situation also determines the r-situation.The graphic statement about the role visual relation“action-action” (“simultaneously”) is shown in Fig. 8(a).Since this is the relation of the objects of the experts’inner world, it is depicted in a circle. Understanding thesymbol “o-situation” implies imaginative representationof the timeline and the vertical line-mark on this timeline,symbolizing the current time and crossing all actions per-formed in the control object simultaneously. At the sametime, the left side of the action-rectangle symbolizes theactual start time of the action, and the right vertex of theaction-triangle symbolizes the planned (estimated) endtime of the action.
The entry (assignment) of an action to a graphicstatement about an o-situation is calculated as follows.The action symbol is placed in the graphic statementif the actual start time of the action is less or equal tothe current time and the estimated end time is greaterthan or equal to the current time. The arrangement ofthe symbols in the graphic expression is ordered relativeto the timeline. It’s possible to move along the circle,from left to right a small round spot, symbolizing thetime flow.
VI. DESIGNING VISUAL-SYMBOLIC STATEMENTSABOUT THE STATE OF THE VISUAL CONTROL OBJECT
Definition 3. The state of the control object S(t) at thetime moment t is a set of parameters, essential for the
problem being solved at the moment t, qualitative fuzzycharacteristics of resources and operations, o-situationsand r-situations, at that the first is considered in thecontext of the production structure, and the second isconsidered in the context of the spatial structure of thecontrol object.
If in the case of r-situation the accents are made onthe mapping of the role spatial relations “resource –resource”, then in the graphical statement in Fig. 8(b)the accents are shifted to the role relations “property –resource”. Since this view relates to the expert’s innerworld, the circles are used to symbolize. There are fourof them. Between the central and the next external inrelation to it circles there are graphical statements aboutthe properties of static resources available in the graphicstatement about the current r-situation. When solvingproblems in the control phase, i.e. when standards forthe properties’ values are given, going beyond the limitsof standards can be symbolized by a change in the colorof triangles. In this case, the color of the central circle cansymbolize integrated either the norm in properties, or thedegree of deviation of the properties of static resourcesfrom standard values.
Between the outer and following to the center circlesthere are graphical statements about the properties ofdynamic resources available in a graphic statement aboutthe current r-situation (there are usually more dynamicresources). When solving problems in the control phase,i.e. when standards for the properties’ values are given,going beyond the limits of standards can be symbolizedby a change in the color of triangles. In this case, thecolor of the ring between the second and third circlesfrom the center can symbolize integrated either the normin properties, or the degree of deviation of the propertiesof dynamic resources from standard values. To giveimaginative character it is possible to specify a rotationfor an outer star-shaped figure (or move a small circularspot around the outer circle), which will symbolize itsrelation to the properties of dynamic elements and thetime flow.
In Fig. 8(c) a complex graphical statement on the stateof the control object at the time t is shown, implementedusing the tool “Graphite" for synthesis of functional hy-brid intelligent systems (FHIS). This statement composedof the graphic statement about the set of parametersessential for the problem solved at time t, qualitativefuzzy characteristics of resources and operations, as wellas graphic statements about the o-situation and the r-situation.
As shown in [19] operator focuses on the upper leftquadrant of the working window, so it is recommendedto place the schematized image of the current o-situationin the upper left part of the working field and theschematized image of the predicted o-situation in theupper right part. The left and right lower parts of the
143
working field are for schematized images of the currentand forecast weather conditions, respectively. In thecenter there is the graphic statement about the r-situation.When displaying the r-situation on a small scale, visualprimitives depicting power lines “merge” into solid blue(gray in grayscale) lines, the power flow along which isshown by moving red (dark gray in grayscale) segments.
Laboratory studies of the developed tool prototype forFHIS with heterogeneous visual field synthesis showedthat specialists who know the subject area of power gridmanagement well, but do not have programming skills,thanks to the presence of visual designers, successfullyconstruct models of the control object, as well as schema-tized images of its states and situations. The experienceof developing FHIS makes it possible to speak of asignificant reduction in the time taken to create themusing the “Graphite” tool.
FHIS developed using the “Graphite” tool will allowto take into account the dynamic nature of complexproblems and to synthesize an integrated method, whichis relevant at the time of problem solving over het-erogeneous visual field. Such FHIS can manage theimitation process, activating the mechanisms of visual-spatial, figurative thinking, when there is a significantuncertainty. These mechanisms allow the user to “see”an approximate solution of the complex problem or itssubtasks, which can later be substantiated and refined bylogical and mathematical reasoning methods.
VII. CONCLUSION
The symbolization and schematization of situational rela-tions and the state of the control object recreate operationallydeformed problem situations, they’re qualitatively better per-ceived by man, and form reference circuits for analyzing andconstructing the new, complex. Drawing, and then understand-ing and rational awareness of the problem with the methods foractivating the right-hemisphere thinking mode, harmonizing ofthe formless, seeing the vague context of the problem situationare the conditions for sudden, relevant decisions and actions ofvisual management.
Functional hybrid intelligent systems, the architectures andmechanisms of which implement the grammar of the visualmetalanguage, will significantly reduce the workload of theoperational and technological personnel, because visual-spatialthinking reflects the world in the fullness of a person’s mind,when one sight is enough to understand the conditions of aproblem situation in the control object and to assess the degreeof risk of continuing of abnormal behavior.
ACKNOWLEDGMENT
The reported study was funded by RFBR according to theresearch project No. 19-07-00208A.
REFERENCES
[1] A.V. Kolesnikov, Gibridnye intellektual’nye sistemy. Teoriya i tekhnologiyarazrabotki [Hybrid intelligent systems: theory and technology of develop-ment], Saint Petersburg: Izdatel’stvo SPbGTU, 2001. (in Russian).
[2] A.V. Kolesnikov, I.A. Kirikov, and S.V. Listopad, Gibridnye intellek-tual’nye sistemy s samoorganizatsiey: koordinatsiya, soglasovannost’, spor[Hybrid intelligent systems with self-organization: coordination, consis-tency, dispute], Moscow: IPI RAN, 2014. (in Russian).
[3] T. Ohno, Toyota production system: beyond large-scale production, Cam-bridge, Massachusetts: Productivity press, 1988.
[4] D. Sibbet, Visual leaders: new tools for visioning, management, andorganization change, Hoboken, New Jersey: Wiley, 2013.
[5] T.S. Stepchenko, Lean-tekhnologii v upravlenii predpriyatiyem[Lean-technologies in enterprise management]. Available at:http://sovman.ru/article/5508/ (accessed 25.12.2018) (in Russian).
[6] Vizual’noye upravleniye: operativno i effektivno [Visualmanagement: quickly and efficiently]. Available at:http://publicatom.ru/blog/SAES/13746.html (accessed 25.12.2018)(in Russian).
[7] Poisk [Search]. Available at: http://poisk-company.ru/ (accessed25.12.2018) (in Russian).
[8] NTK Interfeys [NTK Interface]. Available at: http://iface.ru (accessed25.12.2018) (in Russian).
[9] L. Wittgenstein, Tractatus logico-philosophicus, London: Kegan Paul,Trench, Trubner, 1922.
[10] M.M. Lauzen, and D.M. Dozier, “Issues management mediation of linkagesbetween environmental complexity and management of public relationsfunction” in Journal of public relations research, 1994, Vol. 6, pp.163–184.
[11] J.F. Coates, V.T. Coates, J. Jarratt, and L. Heinz, Issues management: howyou can plan, organize and manage for the future, Mt. Airy, MD: Lomond,1986.
[12] A.I. Uyomov, Veshchi, svoystva, otnosheniya [Things, properties, rela-tionships], Moscow: Izdatel’stvo Instituta filosofii AN SSSR, 1963. (inRussian).
[13] I.A. Kirikov, A.V. Kolesnikov, and V.F. Ponomarev, “Ob odnom podkhodev semioticheskom modelirovanii sostoyaniya transportnykh system [Onone approach in semiotic modeling of the state of transport systems]” inSbornik “Voprosy kibernetiki” [Proceedings “Questions of cybernetics”],Vol.68, Moscow: Izdatel’stvo Nauchnogo Soveta po kompleksnoy prob-leme “Kibernetika" AN SSSR, 1980, pp. 109-130. (in Russian).
[14] F. Kast, and J. Rosenzweig, Organization and management. A system andcontingency approach, New York, 1981.
[15] H. Koontz, and S. O’Donnell, Management: a systems and contingencyanalysis of managerial functions, New York: McGraw-Hill, 1976.
[16] P.R. Lawrence, and J.W. Lorsch, Organization and environment, Home-wood, Ill.: R. D. Irwin, 1969.
[17] J. Thompson, Organizations in action, New York, 1967.[18] D.A. Pospelov, Situatsionnoye upravleniye: teoriya i praktika [Situational
management: theory and practice], Moscow: Glavnaya redaktsiya fiziko-matematicheskoy literatury, 1986. (in Russian).
[19] D.A. Oshanin, Predmetnoye deystviye i operativnyy obraz: izbrannyyepsikhologicheskiye trudy [Subject action and operational image: selectedpsychological works], Moscow, Voronezh: Izdatel’stvo MPSI, Modek,1999. (in Russian).
ВИЗУАЛЬНЫЙ СОБЫТИЙНО-СИТУАЦИОННЫЙПОДХОД ИНФОРМАЦИОННОЙ ПОДГОТОВКИ
РЕШЕНИЙ И ОПЕРАТИВНО-ТЕХНОЛОГИЧЕСКОГОУПРАВЛЕНИЯ СЛОЖНЫМИ ДИНАМИЧЕСКИМИ
ОБЪЕКТАМИ
Колесников А.В.
Оперативная работа с образом непосредственно не вос-принимаемого объекта – трудная задача оператора. Приме-няемые способы отображения не релевантны ментальномуобразу, мешают оператору выполнять действия в уме, спо-собствуют ошибкам интерпретации данных. Изменить поло-жение дел предполагается в рамках визуального событийно-ситуационного субъектно-ориентированного подхода, за-ключающегося в компьютерной имитации полиструктурно-го, многослойного по конкретности и наглядности отобра-жения. В таком подходе предметное действие рассматри-вается как упорядоченная последовательность двухзвенныхпреобразований субъектом деформируемого, ограниченногообъема смешанной перцептивной и предметно-схемной ин-формации о ситуациях и состоянии объекта, в форму целесо-образного, регулятивного воздействия на него. Результатомреализации данного подхода должен стать новый класссистем искусственного интеллекта – когнитивные гибридныеинтеллектуальные системы.
Received 09.01.19
144
Information structures in the framework ofinformation warfare – ontology approach
Peter GrabustsRezekne Academy of Technologies
Rezekne, [email protected]
Abstract—The concept of information warfare involvesthe use of information and communication technology toachieve an advantage over a potential enemy. The goal is totake decisions against their interests in the interests of theirenemies. Information structures are treated as systems thatprocess various types of information, provide its storageand access to users. Such structures may include neuralnetworks, self-learning systems, etc. They must be preparedto train, respond to threats and ensure the safety of theirexistence, which is topical during the modern informationwarfare. In this paper, the theoretical aspects related to thesecurity of information systems from the point of view of thesystem theory and ontology approach will be considered.Knowledge base for information structures can be elementsof artificial intelligence, which must be secured againstexternal threats. Ontologies have gained increasing interestin the computer science community and their benefits arenow recognized for different applications.
Keywords—artificial neural networks, cyberwar, informa-tion structures, information warfare, neural networks
I. INTRODUCTION
The information warfare has always existed - betweenseparate individuals, groups, races, religions, countries,cultures, civilizations. It is always the forerunner anddriver of various wars. H. Lasswell [1] can be calledthe information warfare theorist of the first half of the20th century. He actively used the methods of social psy-chology, psychoanalysis in the study of political behaviorand propaganda, identifying the role of mass media inthe course of information warfare of various states inthe world for power. He identified four main functionsof mass media:
• Collecting and spreading of information.• Selection and commenting of information.• Public opinion formation.• Spread of culture.It is obvious that all these functions are active com-
ponents of the information warfare.The strategy of conducting information warfare by pur-
poseful influence on public opinion presupposes knowingthe moods of all social and ethnic groups, knowingthe real state of things. Hence, on the one hand, infor-mational and psychological impact through all possiblechannels, and on the other hand, a thorough study ofpublic opinion, that is, the identification of the reaction
— the relationship of the elite and the population toinformational and psychological influences, in order tomake adjustments to the impact parameters.
In order for the public to survive in the context of in-formation warfare, it needs to understand the informationstructures and their ability to oppose the impact of theinformation warfare.
The information is tried to be stored so that it can beeasily navigated, that is to quickly find the desired infor-mation element. Therefore, the information is structured,that is, it is written in a definite scheme. Informationstructure (IS) is now the most common term for thoseaspects of a sentence’s meaning that have to do with theway in which the hearer integrates the information intoalready existing information.
An information system is a system that provides:receiving input data; processing of the data; giving out aresult or changing its external state.
An information warfare between two information sys-tems is an open and hidden purposeful informationinfluence of systems on one another with the purposeof getting a certain win.
Information impact is carried out with the use ofinformation weapons, i.e. such means that allow theconceived actions to be carried out with the transmitted,processed, created, destroyed and perceived information.
The aim of the work is to explore the possibilities ofontologies to describe the information structures in caseof danger.
II. THE CONCEPT OF INFORMATION WARFARE
The term “information warfare”, as the 4th generationwar, appeared in the late 80s and very quickly gainedpopularity. So in the beginning of the 90s, the firsttheoretical and later practical works appeared, wherevarious definitions of the “information warfare” weregiven.
Nowadays, the term “cyber war” is used in parallel,which is often endowed with content and meaningsattributed to “information warfare”.
The first profound definition of the term “informationwarfare” was given in the 1996 report of the AmericanRAND Corporation “Strategic Information Warfare andthe New Face of War” [2]. According to it: "Information
145
warfare is a war in the information space”. That is, anew information space is added to the 3 military spaces(land, naval and air) existing at that time.
Subsequently, in the joint document developed bythe headquarters of “Joint Doctrine for Information Op-erations" [3] the definition of “information warfare”,as information operations - conflict in which criticallyand strategically important resource is information thatis to be mastered or destructed was given. This is amultidimensional concept, which is only one aspect,the measurement of which is purely military. The term“information operations” makes it possible, more pre-cisely than the traditional term “information warfare”explore the place and role of information confrontationas components of global confrontations.
There are many other definitions, both official and non-official. According to the work “Information Warfare andSecurity” D. E. Denning said [4]: “Information warfareis a set of operations that have the aim or to exploit theinformation resources". But in the work of G. J. Stein"Information Warfare" [5]: "Information warfare - is theuse of information to achieve our national goals."
The most profound definition of “information warfare”was proposed by the American theorist M.C. Libitsky inhis work “What Is Information Warfare?” dated 1995,where he identified 7 types of information wars [6]:
• Military confrontation for monopolizing command-control functions.
• Confrontation of intelligence service and counterin-telligence.
• Confrontation in the electronic sphere.• Psychological operations.• Organised spontaneous hacker attacks on informa-
tion systems.• Informational-economic wars for controlling the
trade of information products and monopolizingthe information that is necessary to overcome thecompetitors.
• Cybernetic wars in virtual space.
Information warfare can be used among the militaryand among civilians. One of the types of informationwarfare or a set of activities can be used for this purpose.The types of information opposition include:
• Information warfare on the Internet - different andoften contradictory information is offered, which isused to confuse the enemy.
• Psychological operations - the selection and de-livering of such information, which sounds like acounter-argument on the mood that exists in society.
• Disinformation - the promotion of false informationin order to direct the opponent side on the wrongtrack.
• Destruction - the physical destruction or blocking ofelectronic systems that are important to the enemy.
• Security measures - strengthening the protectionof the resources in order to preserve plans andintentions.
• Direct information attacks - confusion of false andtruthful information.
Information warfare can be carried out both within thestate and between different countries. The effectivenessof information warfare depends on well-composed cam-paigning, based on the feelings and desires of membersof society.
The essence of the information warfare is to influencethe society through information. The signs of informationwarfare include:
• Restriction of access to certain information: theclosure of web resources, television programs.
• Creating a negative background on specific issues(fake news).
• Spreading of forced information in various spheresof society.
III. TENDENCES OF INFORMATION WARFARE
Information warfares follow the entire history ofmankind. Propaganda can be considered the first versionof the information warfare. French sociologist J. Ellul[7] offered to differentiate vertical and horizontal propa-ganda. Vertical - this is a classic version of propaganda- information flows from top to bottom with a passiveresponse from the audience.
Horizontal propaganda is realized in the group, anddoes not come from above. In this situation, all par-ticipants are equal. Today’s business actively uses pro-paganda impact methods under other names - publicrelations and advertising.
G. J. Stein publishes the study “Information Warfare”[5], where he emphasizes that information warfare dealswith ideas. Regarding to more specific aims, he statesthe following: “The goal of the information warfare isthe human mind, especially the one that makes the keydecisions of war and peace, and the one that makesthe key decisions about where, when and how to applythe potential and opportunities that are in their strategicstructures”.
In his book “War and anti-war", A. Toffler givesexamples of what is most often used to influence others[8]:
• Accusations of atrocities.• Bid hyperbolization.• Demonization and dehumanization of the opponent.• Polarization.• Divine sanctions.• Meta-propaganda, which discredits the propaganda
of the other side.J. Arquilla [9] has formulated the rule: only the net-
work structure can work effectively against the networkstructure, therefore hierarchical structures that belong to
146
the state will always lose to the network ones. Arquillahas formulated the following three rules for this fight:
• Hierarchies find it difficult to fight networks.• You need networks to fight with networks.• Those who master the first network forms will have
significant advantages.Today, there are many ways and methods of infor-
mation warfare. The author distinguishes software andmedia.
Software means can be classified according to the tasksperformed with their help on means collecting infor-mation, means of distorting and destroying information,and means of influencing the functioning of informationsystems. Some means can be universal and used bothto distort or destroy information, and to influence thefunctioning of information systems.
The main methods and techniques of using informationweapons can be:
• Damage to individual elements of the informationinfrastructure.
• The destruction or damage to the information andsoftware resources of the opponent, overcomingprotection systems, the introduction of viruses, tro-jans and logic bombs.
• Impact on software and databases of informationsystems and control systems with the aim to distortor modify them.
• Capturing media channels in order to spread dis-information, rumors, demonstrate power and bringtheir demands.
• Destruction and suppression of communicationlines, artificial overloading of switching nodes.
• Impact on computer equipment in order to disablethem.
The policy of purposeful influence on public opinionpresupposes knowing the mood of the broad massesof the people, knowing the real state of things. Fromhere, on the one hand, informational and psychologicalimpact through all possible channels, and on the other -a thorough study of public opinion.
IV. INFLUENCE OF INFORMATION WARFARE ONINFORMATION STRUCTURES
The information weapons have a direct relation tothe algorithms [10]. Therefore, any system capable ofprocessing the given algorithm by input data can becalled an information system-the object of an informationwarfare.
One of the key questions leading to the indecidabilityof the problem of winning an information warfare is thefollowing: “Is the information system able to determinethat an information warfare has been launched againstit?”
Why is it necessary to protect the information structurefrom information? Because any information entering
the system inevitably changes the system. Purposefulinformation impact can lead to irreversible changes andself-destruction [10].
Therefore, information warfare is nothing but obviousand hidden targeted informational effects of systems oneach other in order to get a certain gain.
The use of information weapons means the supply tothe input of the information self-learning system of sucha sequence of input data that activates certain algorithmsin the system.
It can be concluded, that information weapon primarilyis an algorithm. To use an information weapon is toselect the input data for the system in such a way thatcertain algorithms are activated in it, and in the case oftheir absence, activate the algorithms for generating thenecessary algorithms [10].
Further, we talk about information structures - trainingsystems - in the simplified assumption it could be artificalneural network (ANN) and social networks. It is assumedthat an information structure is a knowledge carrier andknowledge of an information system is expressed throughits structure. Then, to evaluate the amount of informationperceived by the system, it is logical to use such aconcept as the degree of structure modification by theinput data.
It can be said that the information structure is resistantto external influences if the number of its elements doesnot experience sharp fluctuations from these influences.
Artificial neural networks in general can not be con-sidered as stable information structures. It is connectedwith various training algorithms that work mostly on the"black box" principle, which can make them vulnerableto various external threats.
Artificial neural networks are considered to be apopular approach to machine learning and perception.Traditionally, they are attributed to the properties of self-learning, self-organizing, having ability to process figu-rative information in oppose to conventional algorithms,which are also traditionally considered to be rigidlydefined, untrained, and intended for processing symbolicinformation.
The more complex the network, the more parametersit contains, the more data is required for its training. Usu-ally we do not understand what connection the trainedneural network has with the simulated phenomenon. Itis unclear why it works and we can not predict in whichcases it can fail.
The issue of Artificial Intelligence (AI) limiting hasbeen raised in recent years [11], [12]. An AI box is ahypothetical isolated computer system where a possiblydangerous AI is kept constrained in a “virtual prison"and is not allowed to manipulate events in the externalworld. Such a box would be restricted to minimalistcommunication channels. Unfortunately, even if the boxis well-designed, a sufficiently intelligent AI may nev-
147
Figure 1. Potential reaction structure in case of information warfare attack
ertheless be able to persuade or trick its human keepersinto releasing it, or otherwise be able to “hack" its wayout of the box [11].
The author presents his viewpoint on AI as the pro-tection of information structures in the context of infor-mation warfare. In the context of information warfareagainst a certain AI system (ANN or social networkbased on it) a certain threshold is set up which, appar-ently, should be calculated by some methodology, takinginto account the various activities within the frameworkof the system (fake news, social surveys, etc.). Theimportance of the problem must be taken into accountby the system’s developer (corporation) and, in case ofa critical situation, by the government.
In any case, the system should have a developedmechanism that could be called a trigger, which shouldrespond to an extraordinary intrusion into its structure inthe context of the information warfare. At the same time,the system is learning, re-learning, and self-learning.If, in case of an information warfare attack againstthe information structures the trigger had to work, foursituations would be possible (see Fig. 1):
• Trigger “ON” – the self-destroyed mechanism islaunched – the network activity is paralyzed, linksare destroyed. The AI box protocol is interrupted.
• Trigger “OFF” – the attack is treated as false alarmsand the system continues to work in the previousmode under the AI box protocol.
• Trigger “NEUTRAL” – the attack is treated as anunknown alert and the system continues to work inthe previous mode under the AI box protocol, but byintensifying the analysis of the causes of the attackand trying to identify and prevent future threats.
• Trigger “COUNTERATTACK” – self-learning al-lows the system to exit the AI box protocol frame-work and the effects are not predictable.
The author did not find a formal description of the ISprotection mechanism in the literature available, thus hasoffered own concept in figure 1.
The paper also looks at considers the decision-makingalgorithms in trigger management.
V. ONTOLOGY POSSIBILITIES
In recent years the development of ontologies is formaldescription of the terms in the domain and the rela-tionships between them that moves from the world ofartificial intelligence laboratories to desktops of domainexperts [13]. In the World Wide Web ontologies havebecome common things. Ontologies on the net rangefrom large taxonomies, categorizing Web sites, to cat-egorizations of products sold and their characteristics. Inmany disciplines nowadays standardized ontologies arebeing developed that can be used by domain experts toshare and annotate information in their fields.
The philosophical term “ontology” is known for a longtime, but at the end of the last century, this conceptwas rethought with regard to knowledge engineering. Theclassic definition of an ontology in modern informationtechnologies: “An ontology - a formal specification ofa conceptualization that takes place in a context of thesubject area” [14].
Informally, an ontology is a description of the viewof the world in relation to a particular area of interest.This description consists of the terms and rules for theuse of these terms, limiting their roles within a specificarea. Formally, ontology is a system consisting of a setof concepts and a set of statements about the conceptson the base of which you can build up classes, objects,relations, functions, and theories.
It is accepted that an ontology is a system of conceptsof a subject area, which is represented as a set of conceptslinked by different relations to determine the field ofknowledge. The formal structure of the ontology is an
148
advantage for the quality of the method of knowledgerepresentation.
On a formal level, an ontology is a system consistingof a set of concepts and a set of statements aboutthese concepts, on the base of which we can buildclasses, objects, relations, functions and theory. The maincomponents of the ontology are classes or concepts,relations, functions, axioms, examples.
There are many and different definitions of ontologies,but the following definition has recently been acceptedas generally recognized: “An ontology is a formal ex-plicit specification of a shared conceptualization” [14].Ontologies are often equated with taxonomic hierarchiesof classes. It can be said that the purpose of ontology isto accumulate knowledge in a general and formal way.
Ontologies can be classified in different forms. Oneof the most popular types of classification is offered byGuarino, who classified types of ontologies according totheir level of dependence on a particular task or point ofview [15].
• Top-level ontologies - describe the most generalconcepts that do not depend on the subject areas.
• Domain-ontologies - formal description of the sub-ject area, used to clarify the concepts defined in themeta-ontology and defines a common terminologybase of subject area.
• Task ontologies - an ontology that defines a commonterminology base, related to a specific task.
• Application ontologies - are often used to describethe outcome of actions performed by the objects ofsubject area or the problem.
The simplest model of ontology with relations isusually based on a class-subclass relationships. Suchmodels are often called taxonomies - hierarchies of con-cepts towards investments. Thus, the aim of building anontology is a representation of knowledge in a particularsubject area.
Developing framework Protege OWL tool is used forconstruct this concept [16].
Protege is an ontology and knowledge base editor.Protege is a tool that enables the construction of domainontologies, customized data entry forms to enter data.Protege allows the definition of classes, class hierarchies,variables and the relationships between classes and theproperties of these relationships.
Protege is a special tool, which is thought to createand edit ontology, but OWL (Web Ontology Language)is a language through which it is possible to define theontology. OWL ontology may include descriptions ofclasses, their characteristics and their instances. OWLformal semantics describes how, using these data getinformation which was not openly described in ontology,but which follows from the data semantics. Protege is afree open-source platform, which contains special tool kitwhich makes it possible to construct domain models and
knowledge-based applications based on ontologies. InProtege environment a number of knowledge-modelingstructures and actions that support ontology creation,visualization and editing of different display formats areimplemented.
The development of ontologies with Protege beginswith the definition and description of the classes hierar-chy, after that the instances are assigned of these classesand different type of relationships (properties in Protege)in order to put more meaningful information within theontology.
The author has previously carried out the research onthe ontology-based risk analysis system concept devel-opment [17].
The example of the given ontology is of an illustrativenature, showing a possible ontology in case of any threatsto information structures.
Unfortunately, the author could not find examplesof similar ontologies in the protection of informationstructures, thus the author provides own solution.
The following class hierarchy is defined (see Fig.2).The top level of ontology is the Attack-detection class.
This is an abstract class, which includes all the mainclasses of the subject area:
• Government or corporation.• Situation threshold.In the class Situation analysis the members Trigger
ON, Trigger OFF, Trigger Counterattack and TriggerNeutral are included.
After rating all risks, the situation is analyzed accord-ing to Figure 1. An ontology defined in Figure 2 may beoffered to define threats.
An example of a demonstration shows that with ahelp of Protege an effective ontology description canbe created, but it is a sufficiently laborious process.The author plans to continue the work on the furtherdevelopment of information warfare ontology.
VI. CONCLUSION
Information warfare is a war of technologies; it is awar in which the structures of systems, as carriers ofknowledge, collide. It is necessary to talk about the meth-ods of information warfare because an understanding ofthe techniques of information warfare makes it possibleto transfer it from the category of hidden threats intoexplicit ones that can be dealt with.
Consequences of information warfare:• Death and emigration of part of the population.• Destruction of industry.• Loss of territory.• Political dependence on the winner.• The destruction of the army or the ban on one’s own
army.• Export of the most prospective and high technolo-
gies from the country.149
Figure 2. Example of ontology in case of IS threats
The research presents a description of a potentialcounteraction against the threats of information warfareagainst information systems (AI based on artificial neuralnetworks).
The creation of ontologies is a promising area ofmodern research in information processing, including thesubject of risk analysis in various fields of application.This article examined the ontology prototype approachfor identifying IS threats. The concept of ontology forrisk assessment of IS threats was proposed, some classesand subclasses of ontology under development weredescribed. Thus, ontology becomes a system for storingand managing knowledge.
REFERENCES
[1] H. Lasswell, The Structure and Function of Communication inSociety, The Communication of Ideas, L. Bryson, Ed. Institute forReligious and Social Studies, 1948, p. 117.
[2] R. C. Molander, A. Riddile and P. A. Wilson, Strategic InformationWarfare: a New Face of War, RAND Corporation, 1996. Availableat https://www.rand.org/pubs/monograph_reports/MR661.html (ac-cessed 2018, Nov).
[3] Joint Publication 3-13/Information Operations, Oct. 9, 1998. Avail-able at http://www.c4i.org/jp3_13.pdf (accessed 2018, Nov).
[4] D. E. Denning, Information Warfare and Security, Addison-Wesley,1999.
[5] G. J. Stein, Information Warfare, 1995. Available athttp://www.iwar.org.uk/iwar/resources/airchronicles/stein.htm.(accessed 2018, Nov).
[6] M. C. Libicki, What Is Information Warfare?, National DefenseUniversity Institute for National Strategic Studies, 1995.
[7] J. Ellul, Propaganda: The Formation of Men’s Attitudes, VintageBooks, New York, 1965.
[8] A. Toffler, War and anti-war. Survival at the dawn of the 21stcentury, Little Brown & Co., 1993.
[9] J. Arquilla and D. Ronfeldt, The Advent ofNetwar, RAND Corporation, 2001. Available athttps://www.rand.org/pubs/monograph_reports/MR1382.html.(accessed 2018, Nov).
[10] S. P. Rastorguev, Information Warfare, M: Radio and Communi-cation, 1998 (in Russian).
[11] D. Chalmers, The Singularity: A Philosophical Analysis, Journalof Consciousness Studies, vol.17, no. 7-65, Jan. 2010.
[12] R. V. Yampolskiy, What to Do with Singularity Paradox?, inPhilosophy and Theory of Artificial Intelligence, vol. 5, V. C.Muller Ed. Berlin, Germany: Springer-Verlag, 2013, pp. 397-413.
[13] D. Gaševic, D. Djuric and V. Devedžic, Model driven architectureand ontology development, Springer-Verlag, 2006.
[14] T. R. Gruber, A translation approach to portable ontologies,Knowledge Acquisition, 5(2), pp. 199-220, 1993.
[15] N. Guarino, Formal Ontology in Information Systems. 1st Inter-national Conference on Formal Ontology in Information Systems,FOIS, Trento, Italy, IOS Press, pp. 3-15, 1998.
[16] Protege project homepage. Available at:http://protege.stanford.edu/ (Accessed 2018 Nov).
[17] P. Grabusts, O. Uzhga-Rebrov, Ontology-Based Risk AnalysisSystem Concept. Open semantic technologies for intelligent sys-tems, OSTIS-2017, Minsk, Belarus, pp. 341-346, 2017.
ИНФОРМАЦИОННЫЕ СТРУКТУРЫ ВКОНТЕКСТЕ ИНФОРМАЦИОННЫХ ВОЙН –
ИСПОЛЬЗОВАНИЕ ОНТОЛОГИЙГрабуст П.С.
Понятие информационной войны подразумевает исполь-зование информационных и коммуникационных технологийдля достижения преимуществ по сравнению с потенциаль-ным противником. Информационная война – это манипу-ляция информацией, которой доверяет цель, цель должнапринять решения против их интересов в интересах против-ников. Информационные структуры рассматриваются каксистемы, обрабатывающие различные виды информации,обеспечивающие ее хранение и доступ к пользователям.Такие структуры могут включать в себя нейронные сети,самообучающиеся системыи т.д. Они должныбыть готовымик обучению, реагировать на угрозы и обеспечивать безопас-ность их существования, которая является актуальной вовремя современной информационной войны. В этой работебудут рассмотрены теоретические аспекты, связанные с без-опасностью информационных систем с точки зрения теориисистемы и онтологического подхода. База знаний информа-ционных структур может быть элементами искусственногоинтеллекта, безопасность которых должна быть обеспеченаот внешних угроз. В сфере изучения компьютерных техно-логий интерес к использованию онтологий возрастает, и ихпреимущества теперь признаны для разных приложений.
Создание онтологий является перспективным направле-нием современных исследований по обработке информации,включая тематику анализа рисков в различных областяхприменения. В данной статье рассматривался подход разра-ботки прототипа онтологии для идентификации угроз ИС.Была предложена концепция онтологии по оценке рисковугроз ИС, описаны некоторые классы и подклассы разраба-тываемой онтологии. Таким образом онтология становитсясистемой хранения и управления знаниями.
Received 04.12.18150
Approach to determining the number of clustersin a data set
Ivan Ishchenko, Larysa Globa, Yurii Buhaienko, Andrii LiashenkoNational Technical University of Ukraine ”Igor Sikorsky Kyiv Polytechnic Institute”
Kyiv, [email protected], [email protected]
[email protected], [email protected]
Abstract—For different decision-making systems, theybuild knowledge bases with sets of fuzzy logic rules andwhen constructing these rules on the basis of statisticalinformation, a complex question is the determination of thenumber of clusters. The article is devoted to the analysis ofmethods that allow to automatically determine the numberof clusters and their application in decision-making systems.The analysis conducted helped to distinguish the elbowmethod as the most suitable of all the scanned ones. Thiswas able to find the optimal number of clasters on a testdata set.
Keywords—Clustering, decision-making system, dataanalyses, optimization, fuzzy logic, elbow method, X-meansclustering, silhouette method.
I. INTRODUCTION
Today clustering tasks are relevant for many areas ofactivity. Clustering is intended to divide a set of objectsinto homogeneous groups, and its purpose is to searchfor existing structures. This process is used in com-puter graphics - for image segmentation, for classifyingsearch results, for processing tables and documents, inmarketing - for identifying groups of customers, buyers,and goods for developing promotion strategies. At thesame time, each data domain has its own particular datasets, for example, in technical data collection systemsone has to work with numerical characteristics that havea unique assessment, and for example, when workingwith user / enterprise data, the data has a completelydifferent format. Based on this, different clustering anddata processing algorithms are used.
The problem of determining the number of clusters isone of the main unsolved problems of cluster analysis.The two most used types of cluster analysis proceduresare: hierarchical and iterative. For iterative algorithms,the number of clusters is one of the input parameters ofthe algorithm. For hierarchical procedures, visual anal-ysis of a dendrogram is typical, and the most preferrednumber of clusters is determined from it. [1]
Despite the apparent diversity, so far no universalalgorithm has been found that would be effective for dataof different nature.
Most of the existing methods are based on indexescomparing the degree of ”scatter” of data within clusters
and between clusters, on the calculation of the valuesof heuristic characteristics (stability functions), showingcompliance assigned clusters for selective elements of theset, on the statistics defining the most likely solution,either by estimating the density of distributions. Thedifference between the levels of association, which canbe determined by the dendrogram, is the simplest andmost popular solution.
However, this visual analysis of the dendrogram isextremely difficult when:
1) a large number of objects under consideration;2) implicit expressiveness of the data structure.
For the k-means clustering algorithm, the input param-eter k is used, which determines the number of clusters.The parameter k may be erroneous. It depends on theshape and scale of the distribution of points in the dataset. The number of clusters can be from one to n − 1,where n is the number of objects in the sample. Ie allobjects belong to one cluster or each object is a cluster.[2]
If the number of clusters k from a given data setis not obvious or is not specified by an expert, thereare methods for it that help to make a decision. Theseare direct methods and methods of statistical testing: 1.Direct methods: these are optimization of the criterionwithin cluster sums of squares (the ”elbow” method) orthe average silhouette. 2. Methods of statistical testing:consists of comparing evidence against the null hypoth-esis. An example is the statistics gaps. The method isselected depending on the characteristics of the data set.
One of the important issues dealt with in this paperis a problem of the clusters number definition in theprocess of clustering that based on the statistical data.The number of clusters determines exactly the fuzzylogical rules number that formed the fuzzy knowledgebase. Based on this the correct determination of theclusters number has a significant impact on the qualityof the resulting fuzzy logic rules and consequently thequality of the fuzzy knowledge base in general.
151
II. CLUSTERING METHODS ANALYS
A. The eblow method
The elbow method is based on the use of the per-centage of variance, which is a function of the numberof clusters. The main task is to determine the numberof clusters, such that adding another cluster should notprovide a significant improvement in data modeling.This means that if we build the percentage of varianceexplained by clusters against the number of clusters, thefirst clusters will add a lot of information (they willexplain a large variance), with a subsequent change in thenumber of clusters, the limiting gain starts to decreasesharply, and a clear sectional error appears on the graph.At this point of inflection, the number of clusters isdetermined, and from this the name ”elbow criterion”emerges. But this inflection point may not always beuniquely identified [1]. The percentage of variance isdetermined by the ratio of the variance between groupsto the total variance, also known as the F-test. A slightchange in this method shows the curvature of the intra-group dispersion [2,3].
The optimal number of clusters can be determined asfollows:
1) Calculate the clustering algorithm (for example, k-meansclustering) for different values of k. For example, vary-ing k from 1 to 10 clusters.
2) For each k, calculate the total intracluster sum of thesquare (wss).
3) Get the wss curve in accordance with the number ofclusters k.
4) The location of the bend on the graph is usually con-sidered as an indicator of the corresponding number ofclusters. [3]
Consider a multivariate observation xi =(xi1 , xi2 , ..., xip)′, i = 1, ..., n, containing n independentobjects measured on p variables. For any partition ofthe n objects into g clusters (Pg), denote by Cm theset of objects allocated to the mth cluster and by nmthe number of objects in Cm, m = 1, · · · , g. Denote bydi,i′ the distance between objects i and i′. The sum ofpairwise distances between objects in the mth cluster isgiven by
Dm =∑
i,i′∈cm
di,i′ (1)
For a fixed value of g, define
Wg =
g∑
m=1
1
2nmDm (2)
Note that Wg in (2) is a typical measure of the within-clusters homogeneity associated with Pg , a small valueof which reflects a good fit of a classification to the ”true”cluster structure of data.
In the above definition of Wg , di,i′ can be any arbitrarymeasure of distance. If the squared Euclidean distanceis used, simple mathematical derivation shows that Wg
is monotonically decreasing in g. Hence, Wg is not
informative in choosing the optimal number of clustersby itself. However, for data strongly grouped around Gcenters, it is expected that the value of function Wg
will drop quickly as g increases until it reaches the”true” number of clusters in the data. Intuitively, Wg
will decrease at a much slower rate when g > G sincewith more than G centers, objects belonging to the samecluster will be partitioned.[4,5]
Therefore, an ”elbow” point in the curve of Wg mayindicate the optimal estimate of the number of cluster indata.
In estimating the number of clusters in a data set,methods based on the Wg criterion are aimed at ap-propriately determining the ”elbow” point in Wg , whereWg is sufficiently small. The idea of the gap methodis to compare the curve of Wg from the original datato the curve of the expected) under an appropriate nullreference distribution. The best estimate of the clusternumber is g if Wg falls farthest below the expected curveat g = g. Defining the gap statistic as
Gapn(g) = E∗n log(Wg) − log(Wg) (3)
the estimate g is the value of g which maximizesGapn(g).
An essential step of the gap method is to generatesuitable reference data sets which are used to obtainthe benchmark of the within-clusters dispersion forcomparison. The reference data can be generated byincorporating information about the shape of the datadistribution. By definition, application of the gap methoddoes not depend on the clustering method used. Forexample, Tibshirani et al. implemented the gap methodunder the contexts of both K-means and hierarchicalclustering methods in their research. Simulation studiesshowed that the gap method is a potentially powerfulprocedure in estimating the number of clusters for adata set. Moreover, the gap method has the advantageover most of the other estimating methods that it can beused to test the null hypothesis about homogeneous non-clustered data against the alternative of clustered data.
However, a deficiency of the gap method in findingthe correct number of clusters has been demonstratedin more recent studies. For example, the gap methodfailed to detect the 4-cluster structure in the simulateddata which contain well-separated clusters generatedfrom distinct exponential distributions. In microarraydata analysis, Dudoit and Fridlyand developed the Clestmethod and compared it with several other existingmethods including the gap method. They noted that thegap method tends to overestimate the number of clusters.One possible reason for such a deficiency in using the gapmethod may be because Wg , a statistic summarizing thewithin-clusters homogeneity, is not suitable in measuringthe clustering adequately.[5]
152
B. The silhouette method
The average data power criterion is a criterion forestimating the natural number of clusters. The silhouetteof a data instance is a measure of how closely it iscompared with the data within its cluster and how weaklyit is compared with the data of the neighboring cluster,that is, the cluster, the average distance from which isthe lowest. The silhouette value maximally close to 1means that the base point belongs to the correspondingcluster. When the silhouette value is close to -1, it canbe concluded that the point does not belong to the givencluster.
Optimization techniques, such as genetic algorithms,are useful in determining the number of clusters thatgive the largest silhouette. It is also possible to scalethe data so that the silhouette is maximized with thecorrect number of clusters. In general, this means thatit measures the quality of clustering and determineshow well each object is located within its cluster. Thehigher the average width, the better the clustering. Themean silhouette method calculates the mean observationsilhouette for different values of k. The optimal numberof clusters k is one that maximizes the average silhouettein the range of possible values for k. [3,6]
The algorithm is similar to the elbow method and canbe computed as follow:
1) Compute clustering algorithm (e.g., k-means clustering)for different values of k. For instance, by varying k from1 to 10 clusters.
2) For each k, calculate the average silhouette of observa-tions (avg.sil).
3) Plot the curve of avg.sil according to the number ofclusters k.
4) The location of the maximum is considered as theappropriate number of clusters. [3]
C. Silhouette statistic
Kaufman and Rousseeuw proposed the silhouette in-dex as to estimate the optimum number of clusters inthe data. The definition of the silhouette index is basedon the silhouettes introduced by Rousseeuw, which areconstructed to show graphically how well each objectis classified in a given clustering output.[5] To plot thesilhouette of the mth cluster, for each object in Cm,calculate s(i) asa(i) = average dissimilarity of object i to all other
objects in the mth clusterd(i, C) = average dissimilarity of object i to all other
objects in cluster C, C 6= Cm
b(i) = minC 6=Cmd(i, C)
s(i) = b(i)−a(i)maxa(i),b(i)
The silhouette index, denoted by s ¯(g), is defined asthe average of the s(i) for all objects in the data. s ¯(g) iscalled the average silhouette width for the entire data set,
reflecting the within-cluster compactness and between-cluster separation of a clustering. Compute s ¯(g) for g =1, 2, · · · . The optimum value of g is chosen such thats ¯(g) is maximized over all g:
G = arg maxgs ¯(g).
D. Gap statistic method
The gap statistic has been published by R. Tibshirani,G. Walther, and T. Hastie (Standford University, 2001).The approach can be applied to any clustering method.The gap statistic compares the total within intra-clustervariation for different values of k with their expectedvalues under null reference distribution of the data.The estimate of the optimal clusters will be value thatmaximize the gap statistic (i.e, that yields the largest gapstatistic)[7,8]. This means that the clustering structure isfar away from the random uniform distribution of points.[3]
The algorithm works as follow:1) Cluster the data under investigation for fixed cluster
number, k, where k = 1, 2, · · · . Compute Wk for allvalues of g;
2) Generate B reference data sets in the way describedabove. Cluster each of the B reference data sets andcalculate W ∗b (k), b = 1, 2, · · · ,B and k = 1, 2, · · · .Compute the gap statistic
Gap(k) = (1
B)∑
b
log(W ∗b (k))− log(W (k))
3) Compute the standard deviation
sdk =
[(1
b
)∑
b
(W ∗b (k))− l
2] 1
2
,
wherel = (
1
B)∑
b
log(W ∗b (k))
4) Define sk = sdk
√1 + 1
BThe optimum number of
clusters is given by the smallest k such that
Cap(k) ≥ Cap(k + 1)− sk+1 [9, 10, 11].
III. ESTIMATION OF DIFFERENT METHODS FORFINDING THE NUMBER OF CLUSTERS
A software solution was created to find the numberof clusters in the data sample on different methods. Ob-tained results (Fig. 1):Elbow method: 4 clusters solutionsuggested• Silhouette method: 2 clusters solution suggested• Gap statistic method: 4 clusters solution suggested
The silhouette plots display a measure of how closeeach point in one cluster is to points in the neighboringclusters. This measure ranges from –1 to 1, where 1means that points are very close to their own cluster andfar from other clusters, whereas –1 indicates that pointsare close to the neighbouring clusters.
Gap statistic is a goodness of clustering measure,where for each hypothetical number of clusters k, it
153
Fig. 1. Number of clusters for different algorithms
The silhouette plots display a measure of how close
each point in one cluster is to points in the neighboring
clusters. This measure ranges from -1 to 1, where 1 means
that points are very close to their own cluster and far from
other clusters, whereas -1 indicates that points are close to
the neighbouring clusters.
Gap statistic is a goodness of clustering measure, where
for each hypothetical number of clusters k, it compares two
functions: log of within-cluster sum of squares (wss) with
its expectation under the null reference distribution of the
data. In essence, it standardizes wss. It chooses the value
where the log(wss) is the farthest below the reference
curve, ergo the gap statistic is maximum.
The elbow method maps the within-cluster sum of
squares onto the number of possible clusters. As a rule of
thumb, you pick the number for which you see a significant
decrease in the within-cluster dissimilarity, or so called
‘elbow’.
According to these observations, it’s possible to define
k = 4 as the optimal number of clusters in the data. As we
can see from the three approaches we can to a certain extent
be sure of what an optimal value for the number of clusters
can be for a clustering problem. There are few other
techniques which can also be used.
IV. CONCLUSIONS
The article analyzes a number of clustering algorithms
and their application in decision-making systems. In a
cluster analysis, the fundamental problem is to determine
the value of the number of clusters, which has a
deterministic effect on clusterization results. However, the
limitation in current applications is that there is no
convincingly acceptable solution to the problem with the
best cluster because of the high complexity of real data sets.
Choosing the appropriate clustering method is another
important step in clustering. The k-medium clustering is
one of the most popular clustering technologies used in
practice.
According to the results of the study, it can be
concluded that both the k-medium method and the method
of agglomeration hierarchical clusterization can be
successfully used for clustering in various application areas,
with the results of this clusterization being close. The main
disadvantage of the k-medium method is that it is necessary
to predefine k - the number of clusters and standards, which
is not always possible to make rational. The method is very
sensitive to these initial approximations of the values of the
centers. To eliminate this problem, you can use the method
of gradually increasing the number of clusters.
The disadvantage of elbow and average silhouette
techniques is that they measure only the general
characteristics of clustering. A more complex way is to use
the gap statistics, which provides a statistical procedure for
formalizing a heuristic elbow / silhouette to estimate the
optimal number of clusters.
In this article, we describe various methods for selecting
the optimal number of clusters in the data set. Such
methods include elbows, silhouette, and statistical methods
of rupture.
Future work: Future work is to carry out research on the
possibility and quality of the resulting solution in order to
receive sets of rules for forms of fuzzy knowledge bases for
decision-making systems in technical systems.
REFERENCES
[1] David J. Ketchen Jr; Christopher L. Shook (1996).
"The application of cluster analysis in Strategic
Management Research: An analysis and critique". Strategic
Management Journal. 17 (6): 441&ndash, 458.
doi:10.1002/(SICI)1097-0266(199606)17:6<441::AID-
SMJ819>3.0.CO;2-G
[2] D. Pelleg; AW Moore. X-means: Extending K-
means with Efficient Estimation of the Number of Clusters
(PDF). Proceedings of the Seventeenth International
Conference on Machine Learning (ICML 2000). Retrieved
2016-08-16.
[3] R.C. de Amorim & C. Hennig (2015). "Recovering
the number of clusters in data sets with noise features using
feature rescaling factors". Information Sciences. 324:
Figure 1. Number of clusters for different algorithms
compares two functions: log of within-cluster sum ofsquares (wss) with its expectation under the null refer-ence distribution of the data. In essence, it standardizeswss. It chooses the value where the log(wss) is thefarthest below the reference curve, ergo the gap statisticis maximum.
The elbow method maps the within-cluster sum ofsquares onto the number of possible clusters. As a ruleof thumb, you pick the number for which you see asignificant decrease in the within-cluster dissimilarity, orso called ”elbow”.
According to these observations, it’s possible to definek = 4 as the optimal number of clusters in the data. Aswe can see from the three approaches we can to a certainextent be sure of what an optimal value for the numberof clusters can be for a clustering problem. There arefew other techniques which can also be used.
CONCLUSIONS
The article analyzes a number of clustering algorithmsand their application in decision-making systems. In a clusteranalysis, the fundamental problem is to determine the valueof the number of clusters, which has a deterministic effecton clusterization results. However, the limitation in currentapplications is that there is no convincingly acceptable solutionto the problem with the best cluster because of the highcomplexity of real data sets.
Choosing the appropriate clustering method is another im-portant step in clustering. The k-medium clustering is one ofthe most popular clustering technologies used in practice.
According to the results of the study, it can be concludedthat both the k-medium method and the method of agglom-eration hierarchical clusterization can be successfully used for
clustering in various application areas, with the results of thisclusterization being close. The main disadvantage of the k-medium method is that it is necessary to predefine k - thenumber of clusters and standards, which is not always possibleto make rational. The method is very sensitive to these initialapproximations of the values of the centers. To eliminate thisproblem, you can use the method of gradually increasing thenumber of clusters.
The disadvantage of elbow and average silhouette techniquesis that they measure only the general characteristics of cluster-ing. A more complex way is to use the gap statistics, whichprovides a statistical procedure for formalizing a heuristicelbow / silhouette to estimate the optimal number of clusters.
In this article, we describe various methods for selectingthe optimal number of clusters in the data set. Such methodsinclude elbows, silhouette, and statistical methods of rupture.
Future work: Future work is to carry out research on thepossibility and quality of the resulting solution in order toreceive sets of rules for forms of fuzzy knowledge bases fordecision-making systems in technical systems.
REFERENCES
[1] David J. Ketchen Jr; Christopher L. Shook. The application of clusteranalysis in Strategic Management Research: An analysis and critique.Strategic Management Journal, 1996. Vol 17 (6), pp 441–458
[2] D. Pelleg; AW Moore. X-means: Extending K-means with Efficient Esti-mation of the Number of Clusters. International Conference on MachineLearning no17, 2000. pp 411–416
[3] David J. Ketchen. Determining the number of clusters in a data set. Avail-able at: https://wiki2.org/en/Determining −the −number −of −clusters−in −a −data −set (accessed 2018, Nov)
[4] R.C. de Amorim & C. Hennig. Recovering the number of clusters indata sets with noise features using feature rescaling factors. InformationSciences, 2015. pp 126–145.
[5] Chatti Subbalakshmi, G. Rama Krishnab, S. Krishna Mohan Raoc, P.Venketeswa Raod. A Method to Find Optimum Number of Clusters Basedon Fuzzy Silhouette on Dynamic Data Set. Procedia Computer Science,2015. Vol 46. pp 346–353.
[6] Can, F.; Ozkarahan, E. A. Concepts and effectiveness of the cover-coefficient-based clustering methodology for text databases. ACM Trans-actions on Database Systems, 1990. Vol 15, pp 483.
[7] Charrad, Malika, Nadia Ghazzali, Veronique Boiteau, Azam Niknafs.NbClust: An R Package for Determining the Relevant Number of Clustersin a Data Set. Journal of Statistical Software, 2014. Vol 61, pp 1–36.
[8] Kaufman, Leonard, Peter Rousseeuw. ”Finding Groups in Data: An Intro-duction to Cluster Analysis.” Wiley, 2005. 368p.
[9] Andrzej Piegat. ”Fuzzy Modeling and Control” Heidelberg, Physica–Verlaga Springer–Verlag Company, 2001. 728p.
[10] Mingjin Yan. Methods of Determining the Number of Clusters in a Data Setand a New Clustering Criterion. Nov, 2005 Blacksburg, Virginia. pp.26–27.
[11] Voroncov K.V. Algoritmy klasterizacii i mnogomernogo shkalirovaniya.Available at: http://www.ccas.ru/voron/download/Clustering.pdf (accessed2018, Nov)
ПОДХОД К ОПРЕДЕЛЕНИЮКОЛИЧЕСТВАКЛАСТЕРОВ В НАБОРЕ ДАННЫХ
Ищенко И. А., Глоба Л. С., Бугаенко Ю. М., Ляшенко А. В.
Аннотация – Для различных технических систем принятиярешений создают базы знаний с наборами правил нечеткойлогики. При построении таких правил на основе статистиче-ской информации сложным вопросом является определениеколичества кластеров. Статья посвящена анализу методов,позволяющих автоматически определять количество класте-ров с целью их применения в системах принятия решений.Проведенный анализ математических методов, позволяю-щих автоматически определять количество кластеров припостроении нечеткой базы знаний, а значит и количествонечетких правил, позволяет выделить метод «локтя» какнаиболее подходящий. Метод позволил найти оптимальноеколичество кластеров в наборе тестовых данных.
Received 10.12.18154
Approach to Prediction of Mobile OperatorsSubscribers Churn
Andrii Baria, Larysa Globa, Anastasiia MorozNational Technical University of Ukraine ”Igor Sikorsky Kyiv Polytechnic Institute”
Kyiv, [email protected], [email protected], [email protected]
Abstract—This article presents an approach to the de-scription of machine learning methods for predicting theoutflow of telecom operator subscribers. Describes theparameters characterizing the interaction of the mobileoperator with end users. The parameters that have thegreatest influence on the client’s decision to refuse theservices of a mobile operator have been determined. Theoriginality of the approach lies in the use of such mathe-matical methods that allow you to determine the main setof parameters, due to which specific subscribers are proneto changing the mobile operator. The proposed approachallows you to organize a system in which it is possible todetermine the main parameters characterizing the tendencyof customers to outflow and acting on them using variousmethods to try to increase subscriber loyalty. A comparativeanalysis of the results obtained using the analyzed logisticregression methods, Bootstrap aggregating and randomforest showed that the spread of prediction errors does notexceed 6%. However, the advantage of the random forestmethod is the ability to determine the set of parametersthat make the greatest contribution to making decisions bythe subscriber to change the mobile operator. Therefore,for analyzing customer loyalty, a random forest methodcan be recommend, which showed on the test sample animprovement in the accuracy of the predictions in thesample to 6-7%.
Keywords—telecom operator, churn, machine learning,random forest, prediction mathematical methods.
I. INTRODUCTION
Constantly developing mobile market creates greatcompetition, where subscribers tend to constantly lookfor the most favorable conditions for the provision ofservices by mobile operators. In this regard, operatorsneed to constantly offer the most relevant services foreach subscriber in order to keep this subscriber in theircommunication network. This is because the cost ofretaining the customer is significantly less than attractingnew customers. [1] Telecom operators are constantly an-alyzing the parameters characterizing the use of servicesby the end subscriber to identify those factors that havethe greatest influence on the decision of subscribers toabandon the use of services, as well as to identify thosesubscribers who in the future will be at risk. [2] Thetask is complicate by the presence of a huge amount ofinformation collected, which cannot be process by the”old” methods of information analysis.
To solve such problems can be used different methodsof machine learning, which do not always give goodresults due to the complexity of the data. In this regard,the process of building models is reduce to choosingthe most appropriate machine learning method for aparticular case with a specific data set, which is anontrivial task that requires a specialized approach.
To assess the effectiveness of the use of mathematicalmethods, an experiment was conduct using mathematicalmodeling on the data array of one of the major Ukrainianmobile operators in order to answer the question ”whydo subscribers leave?”
The experiment is divide into 2 stages. The first stageis to determine the reasons why subscribers decide tostop using this or that service. The second stage ofwork consists in compiling a list of subscribers who areprone to care, indicating the percentage of probabilityand the parameters that have had the greatest influenceon decision making.
The structure of the article: Part 2 contains an analysisof work on the outflow of subscribers from companiesproviding various services. Part 3 provides an overviewand analysis of data provided by one of the major mobileoperators. Part 4 describes the metrics for the proposedassessment method with reference to the parameters char-acterizing subscribers who have the greatest influenceon the process of a classifier designing. Part 5 presentsthe results of the prediction of outflow of subscribersfrom the mobile operator. Part 6 includes conclusionsand recommendations for further work.
II. STATE OF THE ART AND BACKGROUNDIn [3] and [4], they talk about solving a similar prob-
lem for companies providing Internet services. For theanalysis, such algorithms as logistic regression method,decision tree, and neural network are considered. Thebest results were obtain on a logistic regression modelwith a prediction accuracy of 89% and a sensitivity of91%.
Alfa-Bank uses the Oracle Exadata storage and pro-cessing platform, the Oracle Big data Appliance andthe Hadoop framework to analyze social networks anduser behavior, assess creditworthiness, forecast customeroutflow, personalize content and secondary sales [5], [6].
155
For the segmentation and management of customerchurn, financial reporting, analysis of feedback on socialnetworks and forums, VTB24 uses Teradata, SAS VisualAnalytics and SAS Marketing Optimizer [7], [8].
All these methods make it possible to obtain a forecastof future trends on the existing data sets, but the defi-nition of the set of parameters that made the greatestcontribution to the current situation remains not fullyresolved.
Research results show that in order to get an adequatesolution to a specific problem, it is necessary to takeinto account the total metrics, the way of collectingdata, their initial structure, ways of processing dataand selecting the best parameters for models. All thisrequires an analysis of mathematical methods and modelsfor solving such problems from the point of view ofaccuracy, completeness and correctness of the resultsobtained. Therefore, the initial stage is to correctly selectthe necessary methods and build the appropriate modelsof data structures that allow you to get the most adequatesolution to a specific task on a specific data set. Theanalysis of the solution of the problem of predicting theoutflow of subscribers from mobile operators allowed usto identify the most promising mathematical methods thatdemonstrate the best results in solving such problems:decision trees, random forest, k-means, naive bays clas-sifier, bootstrap aggregating.
Analysis of the methods of machine learning is carriedout subject to the adoption of a compactness hypothesis,which assumes that classes form compactly localizedsubsets in the space of objects. In general, such ahypothesis is fuzzy, since all classes of recognition apriori intersect in the space of signs. In this case, for theformalization of the concept of ”similarity”, the functionof distance or the metric d (x, y) in the N-dimensionalspace of objects is introduced.
In practical tasks, for the purpose of obtaining thevalues of the function of correspondence, a vector ofsigns of a functional state, which consists of both con-tinuous (quantitative) and discrete (categorical) attributestaking their values from a finite disordered set, areused. With the data within the nominal scale, in whichcategorical signs are measured, no arithmetic operationscan be performed, since all types of numerical processingrelate to the ordering of objects in each class. Bringingcategorical primary attributes to quantitative secondaryby simply numbering the values of the primary attributesrarely results in satisfactory results, since the algorithmswill take into account ordering that has no meaning, soprocessing mixed-type data causes some difficulties andis undesirable.
The disadvantage of algorithms that use a remote met-ric is to ignore additional information that is described bystatistics of qualitative characteristics. For example, fora categorical sign it is possible to calculate its frequency
(number of observations) and fashion (the value that hasthe highest frequency).
Metric algorithms perform a local sampling approxi-mation, in which calculations are delayed until a knowninput object becomes known. Metric algorithms refer tomethods of lazy learning (lazy learning).
The fundamentals of eager learning methods that makeglobal sampling approximation were embodied in thetheory of multivariate statistical analysis and decision-making theory.
The essence of statistical teaching methods is torestore a separate function by minimizing the averagerisk of false decision making. Statistical methods allowconstructing deciding rules in the cases of crossing therecognition classes, which takes place in practical prob-lems of control and management of weakly formalizedprocesses.
The main drawbacks of statistical methods that restricttheir use in practice are the need for large volumes ofstatistics to approximate the probability density distri-bution function, to fulfill rigid conditions for ensuringstatistical stability and homogeneity and high sensitivityto the representativeness of the training samples.
The main disadvantage of the SVM (support vectormachine) method is the limitation of its use for thetasks of analyzing the outflow of subscribers due to thealgorithm’s model due to the ignoring of the a priorisection of recognition classes in the sign space.
The process of constructing models to solve the prob-lem is to choose the most suitable method of machinelearning for a particular case with a specific set ofdata, which is a non-trivial task for which a specializedapproach is required. Forecasting the outflow can beconsidered as a controlled classification problem, inwhich the behavior of the subscriber is used to teachthe binary classifier.
The solution to the problem of customer outflow isreduced to the classification of customers in 2 groups:the customer is prone to outflow or the client is not proneto outflow. If a client falls into the first group, then hemust be influenced by the methods of marketing content.
Forecasting the outflow can be considered as a con-trolled classification problem, in which the behavior ofthe subscriber is used to study the binary classifier.
The most accurate result is to give the random forestalgorithm, because it uses an ensemble of deciduous treesand has the ability to effectively process data with a largenumber of features and classes.
III. SOURCE DATAThe input data gives us different parameters of the
subscribers of the telecommunication network. Parame-ters have been divide into several groups according totheir meaning and purpose of use.
The first group is the main data about the subscriber,which is a general characteristic of the subscriber in
156
the telecommunication network, but do not carry aninformation component for the analysis (see table 1).
Table IDATA ABOUT THE SUBSCRIBER, WHICH IS A GENERAL
CHARACTERISTIC OF THE SUBSCRIBER IN THETELECOMMUNICATION NETWORK, BUT DO NOT CARRY AN
INFORMATION COMPONENT FOR THE ANALYSIS
Title Description
ABON−CODE Hash subscribercode
The subscriber’sstatus indicates
STATUS whether thesubscriber isactive or not
The area in whichthe subscriber
OBLAST uses the servicesof the mobile
operator
The city in whichthe subscriber
CITY uses the servicesof the mobile
operator
The second group provides data about the subscriber’sactivities in the telecommunication network (see table 2).
One of the complexity of working with data is theirdifferent nature, available both numerical and qualitativeparameters that are used for the analysis. Additionaldifficulty is the omission of data. In this case, the correctsolution will fill in the missing data with zeros, as thiswill mean that the subscriber did not use this service orthe absence of a card of other operators at the subscriber.
In the analysis, there may be a situation where indi-vidual data is redundant and does not affect the output,and even adds an additional error, so these data shouldnot be taken into account during the analysis.
Consideration of the available data parameters allowsus to conclude on the different nature of the data and thenon-obviousness of their analysis for operators in orderto assess their performance.
For each sample of the data for which the analysisshould be conduct, it is necessary to determine in detailwith all the features present in the input data set inadvance.
IV. METRICS
In machine learning tasks to assess the quality of themodels and comparison of different algorithms using thefollowing metrics:
– Accuracy,– Precision,– Recall,– Integrated indicator F-measure.
Table IIDATA ABOUT THE SUBSCRIBER’S ACTIVITIES IN THE
TELECOMMUNICATION NETWORK
Title Description
The total numberCN−OMO−6M of outgoing calls
from other mobileoperators
The total numberCN−INTL−6M of outgoing
international calls
OP1−DIFF−A−INC−6M Incoming callsfrom operator # 1
OR2−DIFF−A−INC−6M Incoming callsfrom operator # 2
OP3−DIFF−A−INC−6M Incoming callsfrom operator # 3
Incoming callsOTHER−INTL−DIFF−A−INC−6M from international
numbers
DAYS−INACT−ALL−6 Total non-activedays
Tangent of the tiltMINS−SLOPE of the linear trend
for the number ofminutes weekly
Tangent of the tiltINET−SLOPE of the linear trend
for the number ofminutes weekly
The mean squareINET−STD deviation for
the data volumeweekly
The mean squareREFILL−STD deviation for
replenishmentweekly
”Rise” of theMINS−REG−CONST linear trend for
the number ofminutes weekly
Line up trend liftINET−REG−CONST for volume of data
weekly
”Rise” of the linearREFILL−REG−CONST trend for
replenishmentweekly
Accuracy
A dataset is a dimension table m, which consists ofparameters where i = 1,m. Each i-th parameter in therow pi of the table takes some values. Thus, each rowin the table corresponds to the k-th, where k = 1, n, thestate of the process, which is analyzed.
In the simplest case, such a metric may be the fractionof states of a set of parameters on which the classifierhas made the correct decision.
157
Accuracy = PN
Where, P – number of states of a set of parametersfor which the classifier has made the correct decision,N – the size of the training sample.In this metric, there is one feature that needs to be
taken into account. It consists of assigning all param-eters of the same weight, which may be incorrect ifthe distribution of parameters in the training sample isstrongly shift toward one or more classes. In this case,the classifier has more information about these classesand, accordingly, within these classes, it will take moresolutions that are adequate. In practice, this leads tothe fact that there is an ambiguous definition of metricaccuracy for different classes; the discrepancy can rangefrom 80% in a certain class to about 0% in the other.
The solution to this situation is to teach the classifieron a specially prepared, balanced sample of parametersgroup. The disadvantage of this solution is the loss ofinformation relative to the relative frequency of changesin parameter values.
Precision and recall
Precision and recall are metrics that primarily usealgorithms that require pre-aggregated data as inputdata. Sometimes these metrics are used separately, andsometimes as a basis for derivative metrics, such as theF-measure.
The accuracy of a system within the classroom is theproportion of parameters that really belong to this classamong all the parameters that the system attributed to thisclass. The completeness of the system is the proportionof parameters found by the classifier that really belong tothis class in relation to all parameters in the test sample.
The whole set of samples is divid by the classifier intofour parts:
• TP (True positive) – the samples are clearly identi-fied by the classifier in a positive class.
• FP (False positive) – the samples are not correctlyidentified by the classifier in the positive class.
• TN (True negative) – the samples are clearly iden-tified by the classifier in a negative class.
• FN (False negative) – the samples are not correctlyidentified by the classifier in the negative class.
The dimensions of these parts determine precision andrecall:
precision = TPTP+FP
recall = TPTP+FN
Based on the precision and recall parameters deter-mine the function used to evaluate the effectiveness ofbinary classifiers – F-measure
F-measure
Of course, the higher the precision and recall, thebetter the result. In real life, the maximum values of
precision and recall are not achievable at the same time,so you have to look for a balance. It is advisable to havea universal metric that combines precision and recallinformation to evaluate the matching of the algorithmin order to simplify the decision-making process. In thiscase, the decision process is ask. This metric is the F-measure.
The F-measure is a harmonic mean between the valuesof precision and recall. It should close to zero if theprecision and recall values are approaching zero.
F = 2 ∗ precision∗recallprecision+recall
It is possible to calculate the F-measure by giving adifferent weight to precision and recall if you determinethe priority of one of these metrics during the develop-ment of the algorithm.
Fβ = (1 + β2) ∗ precision∗recall(β2∗precision)+recall
Where β takes values in the range 0 < β < 1, whengiving priority to precision, and when β > 1, priority isgiven to recall. When β = 1, the formula is reduced tothe previous one and there is a balanced F- measure. [9],[10].
The F-measure can be used as a formal metric forassessing the quality of a classifier. It reduces two othermetrics to one number: precision and recall. Such amechanism for assessing the quality of the classifieris much easier to make a decision on the outflow ofsubscribers.
V. MODEL
For solving classification, problems when using versa-tile data, the most popular method of machine learning iscall Random Forest. Random forests are a combinationof random trees, so that each tree depends on the valuesof a random vector taken separately and with the samedistribution for all trees of the forest. The generalizationerror tends to the limit as the number of trees in theforest increases. The error of generalizing the forest oftree classifiers depends on the degree of influence of thegeneralized indicators of individual trees of the forest andthe correlation between them. Internal estimates controlthe error, the degree of influence of generalized indicatorsand correlation; they are used to display the answer to anincrease in the number of functions used in the splitting.Internal assessments are also use to measure variableimportance [11].
Consider the forest construction algorithm.Stage 1. A subsample of a training sample of a given
size is select – a tree is built on it (for each tree there isits own subsample).
Stage 2. To build each splitting (when several edgesgo out from one node) in the tree, we look throughthe maximum number of random signs (for each newsplitting, its own random signs).
158
Stage 3. Choose the best sign and carry out splittingaccording to it (according to a predetermined criterion).A tree is built, as a rule, until the sample is exhausted (un-til representatives of only one class remain in the leaves),but in modern implementations there are parameters thatlimit the height of the tree, the number of objects inthe leaves, and the number of objects in the subsampleat which splitting is performed. This algorithm allowsyou to determine the signs that have made the greatestcontribution, due to the fact that in each node there areknown values of the signs.
We will analyze the quality of the solution to theproblem of predicting churn subscribers using such math-ematical methods:
1) Logistic Regression2) Bootstrap aggregating3) Random ForestWe use the Python programming language and the
Pandas, Numpy, Sklearn, and Matplotlib libraries to buildprediction models for subscribers.
Sklearn library contains ready-made algorithms thatare use to build models. Numpy is use to clear data,structure it and get rid of redundancy. With the help ofMatplotlib, graphing and visual results are obtaine forfurther analysis and decision making for each subscriber.
Traditionally solving the problem of customer churnincludes:
1) Determination of the circle of subscribers, who aresoon ready to abandon the company’s services,
2) Establishing the reasons for the refusal of cus-tomers from the company’s services.
3) Development of cost effective and cost effectivemeasures for their retention.
The results of the prediction regarding the solution ofthe problem of outflow of subscribers, as well as theanalysis of their accuracy and reliability for different al-gorithms are show in Table 3 and in Fig. 1. An analysis ofthe results allows us to draw the following conclusions:mid-term methods for finding the best result, showingthe method of a hypothesis, results for up to 6% of thetotal value of the metrics of the most common methods.It surpasses other methods in accuracy, completeness andaccuracy of prediction.
For the method of machine learning Random Forest,which showed the best results and obtained table 3,showing the influence of the parameters of the telecom-munications network on the outflow of subscribers.
In tab. 4 shows the prediction results for the testsample using the Random Forest method. The numberof correctly and incorrectly estimated data, as well asfirst and second order errors are show.
Definition of important parameters that affect theoutflow of subscribers was performed using the charac-teristic curve (ROC-analysis), which shows the resultsof binary classification, when the model assumes the
Table IIITHE METRIC VALUES OBTAINED FOR THE CONSIDERED METHODS
OF PREDICTING CUSTOMER CHURN
Model precision recall F1 F0.5 AccuracyLogistic 0,709 0,728 0,718 0,713 0,7156
Regression
Bootstrap 0,803 0,75 0,776 0,792 0,7684aggregating
Random 0,815 0,7586 0,786 0,804 0,7776Forest
Table IVSPLIT SET FOR RANDOM FOREST
importance labels
Actual False 2046 463
Actual True 649 1842
probability that the observation belongs to one of twoclasses.
In tab. 5 shows the 5 parameters that have the great-est impact on the values of metrics characterizing theoutflow of subscribers.
Table VTHE IMPORTANCE OF PARAMETERS ON OUTFLOW OF SUBSCRIBERS
No Predicted False Predicted True
14 0.073648 TENURE
23 0.067708 AVG−DAYS−INACT−ALL−6
21 0.057130 DAYS−INACT−ALL−6
22 0.057023 AVG−DAYS−ACT−ALL−6
30 0.047259 MINS−SLOPE
In fig. 2 shows a histogram of the effect of eachparameter on the outflow of subscribers. Based on Fig.2 and tab. 5, you can determine the parameters that havethe greatest impact on the outflow of subscribers. Suchparameters are the duration of using the services of amobile operator, the number of inactive days, the average
We will analyze the quality of the solution to the problem
of predicting churn subscribers using such mathematical
methods:
1) Logistic Regression
2) Bootstrap aggregating
3) Random Forest
We use the Python programming language and the
Pandas, Numpy, Sklearn, and Matplotlib libraries to build
prediction models for subscribers.
Sklearn library contains ready-made algorithms that are
use to build models. Numpy is use to clear data, structure it
and get rid of redundancy. With the help of Мatplotlib,
graphing and visual results are obtaine for further analysis and
decision making for each subscriber.
Traditionally solving the problem of customer churn
includes:
1) Determination of the circle of subscribers, who are
soon ready to abandon the company's services,
2) Establishing the reasons for the refusal of customers
from the company's services.
3) Development of cost effective and cost effective
measures for their retention.
The results of the prediction regarding the solution of the
problem of outflow of subscribers, as well as the analysis of
their accuracy and reliability for different algorithms are show
in Table 1 and in Fig. 1. An analysis of the results allows us
to draw the following conclusions: mid-term methods for
finding the best result, showing the method of a hypothesis,
results for up to 6% of the total value of the metrics of the
most common methods. It surpasses other methods in
accuracy, completeness and accuracy of prediction.
For the method of machine learning Random Forest,
which showed the best results and obtained table 3, showing
the influence of the parameters of the telecommunications
network on the outflow of subscribers. Table 1
The metric values obtained for the considered methods of predicting customer churn
Model precisi
on
recall F1 F0.5 Accuracy
Logistic
Regression
0,709 0,728 0,718 0,713 0,7156
Bootstrap
aggregating
0,803 0,75 0,776 0,792 0,7684
Random
Forest
0,815 0,7586 0,786 0,804 0,7776
In tab. 2 shows the prediction results for the test sample
using the Random Forest method. The number of correctly
and incorrectly estimated data, as well as first and second
order errors are show.
Table 2
Split Set for Random Forest
Predicted False Predicted True
Actual False 2046 463
Actual True 649 1842
Figure 1. ROC Graph for Random Forest
In tab. 3 shows the 5 parameters that have the greatest
impact on the values of metrics characterizing the outflow of
subscribers.
Table 3
The importance of parameters on outflow of subscribers
importance labels
14 0.073648 TENURE
23 0.067708 AVG_DAYS_INACT_ALL_6
21 0.057130 DAYS_INACT_ALL_6
22 0.057023 AVG_DAYS_ACT_ALL_6
30 0.047259 MINS_SLOPE
Figure 2. Histogram of parameter dependencies
In fig. 2 shows a histogram of the effect of each parameter
on the outflow of subscribers. Based on Fig. 2 and tab. 3, you
can determine the parameters that have the greatest impact on
the outflow of subscribers. Such parameters are the duration of
using the services of a mobile operator, the number of inactive
days, the average value of all inactive days, the average value
of all active days and the slope of the linear trend for the
number of minutes weekly. Table 4
Subscribers prone to churn
prob_true
13784 1.000000
Figure 1. ROC Graph for Random Forest
159
We will analyze the quality of the solution to the problem
of predicting churn subscribers using such mathematical
methods:
1) Logistic Regression
2) Bootstrap aggregating
3) Random Forest
We use the Python programming language and the
Pandas, Numpy, Sklearn, and Matplotlib libraries to build
prediction models for subscribers.
Sklearn library contains ready-made algorithms that are
use to build models. Numpy is use to clear data, structure it
and get rid of redundancy. With the help of Мatplotlib,
graphing and visual results are obtaine for further analysis and
decision making for each subscriber.
Traditionally solving the problem of customer churn
includes:
1) Determination of the circle of subscribers, who are
soon ready to abandon the company's services,
2) Establishing the reasons for the refusal of customers
from the company's services.
3) Development of cost effective and cost effective
measures for their retention.
The results of the prediction regarding the solution of the
problem of outflow of subscribers, as well as the analysis of
their accuracy and reliability for different algorithms are show
in Table 1 and in Fig. 1. An analysis of the results allows us
to draw the following conclusions: mid-term methods for
finding the best result, showing the method of a hypothesis,
results for up to 6% of the total value of the metrics of the
most common methods. It surpasses other methods in
accuracy, completeness and accuracy of prediction.
For the method of machine learning Random Forest,
which showed the best results and obtained table 3, showing
the influence of the parameters of the telecommunications
network on the outflow of subscribers. Table 1
The metric values obtained for the considered methods of predicting customer churn
Model precisi
on
recall F1 F0.5 Accuracy
Logistic
Regression
0,709 0,728 0,718 0,713 0,7156
Bootstrap
aggregating
0,803 0,75 0,776 0,792 0,7684
Random
Forest
0,815 0,7586 0,786 0,804 0,7776
In tab. 2 shows the prediction results for the test sample
using the Random Forest method. The number of correctly
and incorrectly estimated data, as well as first and second
order errors are show.
Table 2
Split Set for Random Forest
Predicted False Predicted True
Actual False 2046 463
Actual True 649 1842
Figure 1. ROC Graph for Random Forest
In tab. 3 shows the 5 parameters that have the greatest
impact on the values of metrics characterizing the outflow of
subscribers.
Table 3
The importance of parameters on outflow of subscribers
importance labels
14 0.073648 TENURE
23 0.067708 AVG_DAYS_INACT_ALL_6
21 0.057130 DAYS_INACT_ALL_6
22 0.057023 AVG_DAYS_ACT_ALL_6
30 0.047259 MINS_SLOPE
Figure 2. Histogram of parameter dependencies
In fig. 2 shows a histogram of the effect of each parameter
on the outflow of subscribers. Based on Fig. 2 and tab. 3, you
can determine the parameters that have the greatest impact on
the outflow of subscribers. Such parameters are the duration of
using the services of a mobile operator, the number of inactive
days, the average value of all inactive days, the average value
of all active days and the slope of the linear trend for the
number of minutes weekly. Table 4
Subscribers prone to churn
prob_true
13784 1.000000
Figure 2. Histogram of parameter dependencies
value of all inactive days, the average value of all activedays and the slope of the linear trend for the number ofminutes weekly.
Table VISUBSCRIBERS PRONE TO CHURN
No prob−true
13784 1.000000
12763 1.000000
17293 0.914286
10610 1.000000
10555 1.000000
In table. 6 presents data on the likelihood of sub-scribers to be inclined to outflow and to apply marketingmethods to retain them in the network of the mobileoperator.
CONCLUSIONSThe paper describes an approach to getting information
about the outflow of subscribers of a telecommunicationsoperator. For this purpose, it is propose to use machine-learning methods.
The studies identified a set of subscriber’s telecom-munications network, which is sufficient for the task.These data form the basis of a process of learning andprediction.
The best results showed a machine learning techniqueRandom Forest which indicators are better to 6-7%as compared to the methods of logistic regression andBootstrap aggregating.
The result of solving the problem of outflow is:Identification of the number of subscribers in a short
time ready to abandon the services of the company,Identifying the causes of the failure of customers of
the company’s services.Further studies will be focus on increasing the ac-
curacy and completeness of the predictions, as well ason the development of appropriate measures to retaincustomers.
REFERENCES
[1] Numb: who of mobile operators earned the most. Availableat: https://delo.ua/business/onemeli-kto-iz-mobilnyh-operatorov-zarabotal-bolshe-vsego-337433/
[2] Hastie T., Tibshirani R., Friedman J. The Elements of Statisti-cal Learning: Data Mining, Inference, and Prediction. 2nd ed.SpringerVerlag, 2009. page 746
[3] Arustamov, A. Churn in telecommunication companies. Availableat: https://vdocuments.site/9-548b9954b479594c5f8b4658.html
[4] Leo Breiman. Random Forests. Machine Learning October 2001,Volume 45, Issue 1, pages 5–32
[5] Khan, A. A., J. Sanjay, and M. M. Sepehri. 2010. Applyingdata mining to customer churn prediction in an Internet serviceprovider. Int. J. Comput. Appl. 9(7): pages 8–14.
[6] W. Verbeke, K. Dejaeger, D. Martens, J. Hur, and B. Baesens,”New insights into churn prediction in the telecommunicationsector: A profit driven data mining approach,” Eur. J. Oper. Res.,vol. 218, no. 1, Apr. 2012, pages 211–229
[7] Y. Sasaki,. The truth of the F-measure. 26thOctober, 2007 Available at: https://www.toyota-ti.ac.jp/Lab/Denshi/COIN/people/yutaka.sasaki/F-measure-YS-26Oct07.pdf
[8] Telecom Customer Churn Prediction Models. Available at:https://parcusgroup.com/TelecomCustomer-Churn-Prediction-Models
[9] How Ukrainians choose a mobile operator. Available at:https://biz.nv.ua/experts/skorbota/kak-ukraintsy-vybirajut-mobilnogo-operatora-1930612.html
[10] Berkana, A. What is Big data: Collected all the most importantinformation about big data. Available at: https: //rb.ru/howto/chto-takoe-big-data
[11] J. Franklin, ”The elements of statistical learning: data mining,inference and prediction,” Math. Intell., vol. 27, no. 2, Nov. 2008,pages 83–85
ПРЕДСКАЗАНИЕ ОТТОКА АБОНЕНТОВ ОТОПЕРАТОРОВ МОБИЛЬНОЙ СВЯЗИ
Баря А.Д., Глоба Л.С., Мороз А.М.
Аннотация – В данной статье представлен подход кописанию методов машинного обучения для предсказанияоттока абонентов оператора связи. Описаны параметры, ха-рактеризующие взаимодействие оператора мобильной связис конечными абонентами. Определены параметры, оказыва-ющие наибольшее влияние на решение клиента об отказеот услуг мобильного оператора. Оригинальность подхода за-ключается в использовании таких математических методов,которые позволяют определить основной набор параметров,из-за которых конкретные абоненты склонны к смене мо-бильного оператора. Предложенный подход позволяет орга-низовать систему, в которой возможно определить основныепараметры, характеризующие склонность клиентов к оттокуи воздействуя на них разными методами пытаться повыситьлояльность абонента. Сравнительный анализ результатов,полученных с помощью проанализированных методов ло-гистической регрессии, беггинга и случайного леса пока-зал, что разброс ошибки предсказания не превышает 6%.Однако преимуществом метода случайного леса являетсявозможность определить набор параметров, которые вносятнаибольший вклад в принятие решений абонентом о сменемобильного оператора. Поэтому для проведения анализа, ка-сающегося лояльности абонентов может быть рекомендованметод случайного леса, которыйпоказал на тестовой выборкеулучшение правильности предсказаний по выборке до 6-7%.
Received 10.12.18160
Ontological Approach to Analysisof Big Data Metadata
Julia RogushinaInstitute of Software Systems
National Academy of Sciences of UkraineKyiv, Ukraine
Anatoly GladunInternational Research and Training Center of IT and Systems
National Academy of Sciences of UkraineKyiv, Ukraine
Abstract—Now data becomes the most valuable commod-ity that allows to make informed decisions in various areasof human life. In this article, we look at the features of BigData generated by the Internet of Things (IoT) technology,and also present the methodology for Big Data processingwith use of semantic modeling (ontologies) at all stagesof the Big Data life cycle. Use of Big Data semantic modelallows eliminating such contradictions in these technologiesas the heterogeneity of devices and things that causes theheterogeneity of the data types produced by them.
Machine learning is used as an instrument for analyzes ofBig Data: it provides logical inference of the rules that canbe applied to processing of information generated by smarthome system. In this methodology, the authors proposethe use deep machine learning, based on convolutionalneural networks because this model of machine learningcorresponds to processing of unstructured and complexnature of the IoT domain.
This approach increases the efficiency of IoT Big Dataprocessing and differs from traditional processing systemsby using NoSQL database, distributed architectures andsemantic modeling. We propose the conceptual architectureof the Big Data processing system for IoT and describe it onexample of the NoSQL database for the smart home. Thisarchitecture consists of five independent levels. A combinedapproach of semantic modeling and data mining methodscan be used at each of these levels. Currently, this platformcan be combined with a lot of open source components.
Keywords—Big Data, ontology, metadata
I. INTRODUCTION
"Big Data" is a term that refers to a group of technologiesoriented on obtaining of qualitatively new knowledge fromlarge amounts of data that cannot be handled by traditionalmethods and serve. Exponential growth of data generatedin electronic form and stored in data banks determines theactuality of such technologies.
We can consider some set of data as Big Data (and analyze itwith Big Data technologies) if it has one or more of followingfeatures named 5V: Volume – great amounts of data that requirespecialized means of processing; Velocity – great speed of newdata generation and transformation; Variety – different dataformats and types that complicate data integration; Veracity– messiness or trustworthiness of the data that cannot beconverted into information; Value – big parts of data may arenot useful for any users.
The analysis of large data sets is an interdisciplinary task thatcombines mathematics, statistics, computer science and specialknowledge of the domain. This direction of IT is closely allied
with intelligent information systems (IIS) and applied aspectsof artificial intelligence (II).
For effective practical use of Big Data we need to analyzethem at the semantic level with use of domain knowledge.Today mankind generates more and more Big Data volumes.However, this information has no direct value, but is obtainedonly as a result of data analysis. Obtaining of knowledge fromBig Data uses machine learning (ML) [2] that summarizesexperience of some system stored electronically and tries toimprove the behavior of this system.
ML results are not true but probabilistic and statistic, andtheir quality depends on how much the data processed are closeto those used in practice. This fact defines that selection ofpertinent sets of Big Data is very important step of it’s analysis.Metadata of Big Data can be used as a main information sourcethat characterized the semantics of data content. The majorproblems in Big Data technology [1] deal with semantics are:use for data integration; detection of Big Data sets pertinentto user task; and removal of data ambiguity (for example,homonymy). The solutions of these problems use metadatalinked with Big Data are [2]. Although metadata managementhas been known for decades, but processing of Big Datarequires development of new strategies and approaches.
II. PROBLEM DEFINITION
Method of Big Data metadata analysis allows to selectthe task-pertinent data sets from heterogeneous sources anddata warehouse on semantic level with the help of domainknowledge. The natural language (NL) part of such metadata(unstructured or semistructured annotations, descriptions, etc.)is ambiguous, and this fact causes the need in methods ofambiguity resolution (for example, for homonyms and pol-ysemantic terms). We propose to match such metadata withuser task description by methods of NL analysis enhancedwith Big Data ontology contained domain-specific knowledgefor semantic processing of Big Data metadata and their linkswith the domain concepts. Development a prototype of suchontology is also a part of this work.
III. METADATA USED FOR BIG DATA DESCRIPTION
Metadata is a special kind of information resources, theircreation often requires considerable effort and substantial costs,but they significantly increase the value of the data and provideextended opportunities for their use. Metadata is defined asa structured, coded data that describes the characteristics ofvarious (text, multimedia, etc.) objects that facilitates the identi-fication, detection, evaluation and management of these objects.Metadata describes the meaning and properties of information
161
in order to improve management, understanding, classificationand exploitation of this information.
Metadata used for Big Data description is a data blockphysically joined to Big Data in its storage. This metadataprovides information on the characteristics and structure of BigData set: name; the origin of data, data source information;information about the author and date of the data creation; datasize and format, control total; number of dataset records; imageresolution; a brief description of the data etc. [3]. It is importantto note that all changes of Big Data state initiate changes ofmetadata. The structure, functions and properties of metadatadepend considerably on the particular technological realizationand on the features of the described resources, as well as on thescope and specificity of applications. However, the interpreta-tion of the term "metadata" is not defined unambiguously. Nowspecialists use a lot of different definitions of metadata. Themost significant of them are: metadata is data about data [4];metadata is information that makes the data useful [5]; metadatais machine-processed data that describes some resources, bothdigital and non-digital [6]; metadata is information that impliesits computer processing and interpretation of digital and non-digital objects by people [7]; metadata is structured informationthat describes, explains, indicates location and, thus, facilitatesthe retrieval, use and management of information resources [8];metadata in the Web is semistructured data, usually agreed withthe corresponding models that provide operational interoper-ability in a heterogeneous environment [9].
IV. INTEGRATION OF ARTIFICIAL INTELLIGENCEWITH BIG DATA ANALYSIS
The great number of publications in this sphere shows a highinterest to use of methods of traditional artificial intelligence(AI) and intelligent Web technologies to acquisition of knowl-edge from Big Data. Most often, researchers work in directionof ML use and integration of ontological analysis for variousphases of Big Data analysis to apply the domain knowledge.The Ontology Summit 2017 "AI, learning, reasoning and on-tologies" [10] analyses the use AI methods of for ML, logicalinference and ontological analysis focused on Big Data andintegrates various research approaches in this area divided intosome groups:
• Application of ML for extraction of knowledge and im-proving of domain ontologies – creation and improvementof sufficient domain knowledge (knowledge bases andontologies) about the world for a truly intelligent agent,the use of automation and various ML approaches toknowledge extraction and ontological analysis;
• Usage of domain knowledge to improve results of ML –challenges and role of background knowledge and ontolo-gies in improvement of ML results, the requirements forontologies used in ML for various data sets (in particular,for Big Data);
• Integration of ontological analysis with logical inference– the reasoning techniques and mechanisms oriented onontological knowledge representation in various forms.
Background knowledge in Big Data is processed by onto-logical analysis and logical inference by ML means to preparedata for training and testing (reduction of large, noisy data setsto managed ones) and eliminating the ambiguity of terms.
Before the learning phase of ML we have to define suchinput information:
• Description of solved task;• Target function of ML that depends from objectives of
system’s behavior improvement (for example, increasingof the recognition accuracy, expansion the number or classof identified objects, acceleration of recognition);
• Data source that contains information required for anal-ysis, its type, origin and structure (information receivedfrom the system experience of interaction with one user orwith the entire community of users, information receivedfrom one or more external sources, etc.);
• Methods and means that provide integration of the ob-tained results with the existing knowledge of system.
Quite often Big Data for analysis is obtained from variousexternal sources. Velocity of Big Data analysis depends on theamount of information being processed. So prior filtering ofinformation decreases the time of it’s analysis. For example incase of analysis of the television streams we can analyze notall of them but only the selected part of the programs pertinentto user’s problem.
If we plan to use the external experience presented in BigData then we have to find relevant Big Data sources. We cando it with the help of the metadata that accompanies Big Databy analyzing of metadata semantics. Automatically generatedpart of the metadata does not contain enough information aboutcontent semantics. Available metadata is technical informationthat characterizes the time of the content creation, its volume,formats, etc., but does not relate to the information contentof the data. This makes it impossible to provide a uniformdescription of the data semantics. But a big part of Big Data isaccompanied by annotations or explanations, usually providedin natural language. Required information from Big Data can beretrieved by analysis of their annotations. Therefore, matchingof annotations with task definition determines the pertinenceof certain arrays of Big Data to this task. Big Data annotationfrom metadata is matched with the user’s task description.Such matching is executed on the stage of data retrieval andselection, because direct comparison of Big Data content withthis description inappropriate due to the extremely large volumeand absence of structuring. Various annotations of Big Dataare created in process of it’s storing into the repositories. Alltypes of annotations described Big Data on different levelscan be considered as unstructured or semistructured NL texts.Therefore we apply to them standard tools of NL analysissimilar to the Web search. Unfortunately, in the general casesuch retrieval problem is not solved effectively, and thereforeit is advisable to apply a priori additional knowledge about BigData domain. Analysis of scientific reports and correspondingstandards shows that despite the high interest in Big Data andvariety of technological means for their processing, today anymetadata standards specific to Big Data is not developed. Suchsituation is caused by the complexity and variety of Big Data.
Metadata improves data analysis (OLAP, OLTP, Data Min-ing) by understanding of the data source domain in order toensure adequate computation and interpretation of results. Itprovides the use of general terminology for interaction withuser.
V. METADATA STANDARDS APPLICABLE TO BIGDATA
Taking into account the lack of specific for Big Datastandards for metadata, it is reasonable to analyze the existingmetadata standards used for information that can have 5Vproperties and able to represent the content semantics. Thestandards of ISO/IEC 11179 series define metadata as data thatdefines and describes other data. This means that the metadatais data, and data becomes metadata when they are used inthis way. This occurs in specific circumstances, for specificpurposes, with defined prospects. A set of circumstances, goals,or prospects for which some data is used as metadata is calleda context. Thus, metadata is data in some fixed context.
162
Metadata is stored in some database that is organized withthe use of any formal meta-model that describes metadata.For example, the conceptual model defined in ISO/IEC 11179-3 is a meta-model in this content. A significant part of BigData is multimedia information. We analyze some widelyused standards used for describing of multimedia semantics.Now many various formats for multimedia representation aredeveloped by different software and hardware manufacturers,but there is no unique standard common to everyone, becauseeach manufacturer develops its own convenient approach thatcan subsequently be disseminated. Existing formats for savingmultimedia in electronic form (GIF, TIFF, PIC, PCX, JPEG,PNG, etc.) differ in methods of information compression,encodings types, purpose of use etc.
Much of them are not oriented on describing of multime-dia semantics and deal only with technical characteristics ofmultimedia, and only some of them represent the meaning andsubject domain of data. The Moving Picture Experts Groupfor the Joint Standardization Committee propose a family ofmulti-media standard MPEG [11]. Some of them (MPEG-1 (ISO/IEC 11172), MPEG-2 (ISO/IEC 13818), MPEG-4(ISO/IEC 14496)) deal only with compression of multimediainformation. Other ones describe the semantics of multimediacontent.
Standard MPEG-7 ("Multimedia Content Description Inter-face" ISO/IEC) [12] describes the semantic aspects of multime-dia content with different degree of attention to details. MPEG-7 proposes the fixed set of descriptors for different types ofinformation that formalize the defining of descriptors and theirinterconnections.
Multimedia descriptive differ for various domains and appli-cations because the same content can be described at differentabstraction levels through different properties relevant to thescope of use. For example, a graphic image at the lowestlevel of abstraction can be described by size, number of color,forms and positions of objects, while the upper level willcontain semantic information connected graphical elementswith domain concepts| Usually high-level descriptions of mul-timedia are represented by non-structured or semistructuredNL text. For example, "Red dog named Lada de Mandrakastands near the black car". There may also be intermediatelevels of abstraction. The level of abstraction is related to theway of information obtaining: many low-level properties canbe extracted automatically, while high-level properties requirehuman participation.
MPEG-21 [13] is a "Multimedia Framework" is oriented onfor semantic search. It is developed for representation of contentmanagement infrastructure in a distributed environment. Thisstandard defines the basic syntax and semantics of multimediaelements, dependencies between them and the operations thatthey support. It is serving to establish interoperability betweenmultimedia information resources.
RDF (Resource Description Framework) [14] is a part of theSemantic Web project designed for creating semantic metadatafor various types of information. RDF is intended to standardizethe definition and use of Web metadata resources, but it isalso applicable to the description of Big Data. It uses the basedata model "object – attribute – value". RDF Schema givesa possibility to define a specific dictionary for RDF data andspecify the types of objects to which these attributes can beapplied, that is, mechanism of RDF Schema provides a basicsystem of types for RDF models. RDF standard is extensibleand can specify the structure of the source description by usingand extending the built-in concepts of RDF schemes (classes,properties, types, etc.).
Standards for describing typical resources help to simplify
and unify the creation of meta-descriptions. The most well-known set of elements for metadata creation "Dublin CoreMetadata Elements" [15] can be used for description of theBig Data sets.
VI. BIG DATA ONTOLOGY
Ontologies in knowledge engineering are used for formal anddeclarative description of some domain [16]. A wide range ofontologies available through the Web confirms the popularity ofthis approach among various groups of developers and users ofWeb applications, including Big Data. Such ontologies differ bythe volume, expressive means, purpose, degree of knowledgeformalization, etc. [17]. Domain ontology is the part of thedomain knowledge limited the meaning of terms that do notdepend on changing part of domain knowledge. It determinesthe agreements about domain terms [18].
Big Data ontology contains classes for selection of typicalfor Big Data information objects (video, audio, streamingvideo, semistructured data from sensors) with sets of relevantsemantic properties. Examples of classes are Big Data formatsand source types; examples of properties are geographical lo-cation, time of creation, size, annotation. Big Data ontology al-lows to represent the semantics of links between individual BigData fragments (temporal, geographic, communicational (forexample, information about communications by smartphones),by device identifiers, by subject, by purpose, etc.). It fixes alsothe quality parameters of Big Data such as noise, accuracy,degree of trust to the source, signal quality, completeness, etc.
The individuals of Big Data ontology can be matched withthe individuals of user task ontology to search the pertinentsources for analysis.
To use ontological knowledge for comparing such infor-mation objects as annotations – unstructured NL texts – itis necessary to provide mechanisms for linking elements oftheir content with ontology terms. Such mechanism can usethe task thesaurus (dictionary of the basic concepts of languagelinked with separate words or phrases with certain semanticconnections between them [19]) based on the domain ontology[20]. Task thesaurus is a set of concepts necessary to describeand solve a problem for which the user is trying to findsome information by analysis of some Big Data set. Thesaurusconcepts can be imported from domain ontology. Thesauri areused in semantic markup of NL texts [21]. The similarity ofBig Data annotation and user task is estimated by the semanticproximity between their thesauri.
VII. CONCLUSIONS
The analyzing the existing means of Big Data descriptionshows the lack of generally accepted standards for metadatarepresentation. Therefore, we propose to match the naturallanguage annotations of Big Data with user task with the helpof ontological representation of knowledge about Big Dataand task domain. Prototype of Big Data ontology formalizesinformation about Big Data structure, sources, data sets, etc.and provides filtering of data pertinent to particular user task.This ontology includes elements of standards for descriptionof various information types that can be used for Big Datarepresentation.
REFERENCES
[1] Marz, N., Warren, J. (). Big Data: Principles and best practices ofscalable real-time data systems. New York; Manning PublicationsCo.2015
[2] Bizer C., Boncz P., Brodie M.L, Erling O. The meaningful useof Big Data: four perspectives – four challenges SIGMOD Rec.40 (4) 2012. pp.56–60.
163
[3] Smith K., Seligman L., Rosenthal A., Kurcz Ch., Greer M.,Macheret C., Sexton M., Eckstein A. Big Metadata: The Needfor Principled Metadata Management in Big Data EcosystemsProceedings of the Company DanaC@SIGMOD, Snowbird, UT,USA, 2014. – P. 46-55.
[4] Jeusfeld M.A. Metadata Encyclopedia of DatabaseSystems, Springer, 2009, 3, p.1723-1724.http://www.springerlink.com/content/h241167167r35055/.
[5] Grotschel M., Lugger J. Scientific Information System andMetadata. Konrad-Zuse-Zentrum fur Informationstechnik, Berlin.– http://www.zib.de/ groetschel/pubnew/paper/groetschelluegger1999.pdf
[6] Halshofer B., Klas W. A Survey of Techniques for AchievingMetadata Interoperability ACM Computing Surveys, Vol. 42, No.2, 2010.
[7] Metadata Standards and Applications. Introduction:Background, Goals, and Course Outline. ALCTS. –http://www.loc.gov/catworkshop/courses/ metadatastan-dards/pdf/MSA Instructor Manual.pdf.
[8] Uniform Resource Identifier (URI): Generic Syntax. –http://tools.ietf.org/html/rfc3986 .
[9] Lagose C. Metadata for the Web. Cornell University. CS 431 –March 2, 2005.
[10] Baclawski K., Bennett M., Berg-Cross G., Fritzsche D., Schnei-der T., Sharma R., Westerninen A. Ontology Summit 2017communiqué–AI, learning, reasoning and ontologies. AppliedOntology, 2018, P.1-16. – http://www.ccs.neu.edu/home/kenb/pub/2017/09/public.pdf .
[11] MPEG-21 Multimedia Framework, Introduction, ISO/IEC,http://mpeg.telecomitalialab.com/standards/mpeg-21/mpeg-21.htm.
[12] MPEG-7 Overview, ISO/IEC, 2002. –http://mpeg.telecomitalialab.com/standards/mpeg-7/mpeg-7.htm
[13] MPEG-21 Overview v.4, 2002. –http://mpeg.telecomitalialab.com/standards/mpeg-21/mpeg-21.htm.
[14] Lassila O., Swick R. Resource Description Framework (RDF)Model and Syntax Specification W3C Recommendation. –http://www.w3.org/TR/REC-rdf-syntax.
[15] Dublin Core Metadata Elementshttp://www.faqs.org/rfcs/rfc2413.html.
[16] Gruber T., What is an Ontology? – http://www-ksl.stanford.edu/kst/what-is-an-ontology.html.
[17] Guarino N. Formal Ontology in Information Systems FormalOntology in Information Systems. Proc. of FOIS’98, 1998. – P.3-15.
[18] Gladun A., Rogushina J., Schreurs J. Domain Ontology, an Instru-ment of Semantic Web Knowledge Management in e-LearningInternational Journal of Advanced Corporate Learning (iJAC),V. 5, Issue 4 (2012). – P.21-31. – http://online-journals.org/i-jac/article/view/2288.
[19] ISO 25964-1:2011, Thesauri and interoperability with other vo-cabularies. Part 1: Thesauri for information retrieval / Geneva:International Organization for Standards, 2011.
[20] Gladun, A., Rogushina, J. (2012). Use of semantic web technolo-gies and multilinguistic thesauri for knowledge-based access tobiomedical resources International Journal of Intelligent Systemsand Applications, 4(1), 11.
[21] Gladun, A., Rogushina, J., Valencia-García, R., Béjar, R. M.(2013). Semantics-driven modelling of user preferences for infor-mation retrieval in the biomedical domain Informatics for healthand social care, 38(2), 150-170.
ИСПОЛЬЗОВАНИЕ ОНТОЛОГИЙ ДЛЯАНАЛИЗА МЕТАДАННЫХ BIG DATA
Рогушина Ю. В., Гладун А. Я.
Работа посвящена разработке онтологическихсредств анализа Big Data, к которым невозможноприменить традиционные аналитические подходы, ос-нованные на решениях бизнес-аналитики и системахуправления базами данных.
Авторы представляют метод анализа метаданных,описывающих Big Data, который позволяет выбиратьте блоки информации среди разнородных источникови хранилищ данных, которые пертинентны задачепользователя. Большое внимание уделяется сопостав-лению аннотаций (естественно-языковой части мета-данных) с текстом, описывающим задачу. Предлагает-ся использовать для этого средства анализа естествен-ного языка и онтологию Big Data, содержащую знанияо специфике этой предметной области.
Использование искусственного интеллекта и ин-теллектуальные веб-технологии повышают эффектив-ность всех этапов обработки Big Data. Распознаваниетекстовой части метаданных выполняется на основеонтологии Big Data, которая содержит знания о ихспецифике. Разработан прототип такой онтологии,представлена архитектура интеллектуальной системисопоставления аннотаций Big Data с использованиемтезаурусов.
Received 21.12.18
164
Ontological approach to the automated designof complex systems using aircraft as
an exampleAnastasiya Malochkina
Samara UniversitySamara, Russia
Nikolai BorgestSamara University, ICCS RAS
Samara, [email protected]
Abstract—This article discusses the process of complexsystems automation. The analysis of some existing ap-proaches based on the use of ontologies was made: a Robot-Designer created at Samara University and the DesignCockpit 43®, a compiler using design languages, createdat University of Stuttgart. For the Robot-Designer, theformalization of knowledge, design procedures and oper-ations in the chosen field of knowledge is considered, alongwith semantic and mathematical models. A special placein the creation of a robot designer takes an interface thatprovides the designer with information for making a finaldecision and, if necessary, “explains” the need to choosea specific solution. For the Design Cockpit 43® there is adescription of design languages is given: vocabulary, rules,grammar, structure of information within the language.Graph-based design languages are presented as a methodto encode and automate the complete design process andthe final optimization of the product or complex system.The description and methods of ontology implementationare given. The task is to consider already existing methodsof automation of aircraft design. It affects the formalizationof the design process as such and, accordingly, the stagesat which it is most advisable to automate human activities.Examples show how to put it into practice in the modernproduction of this kind of automation. The results achievedand possible future development prospects are indicated.The relevance of the article is justified by the growinginterest in the automation of design and production sinceit reduces the time for design and reduces the number oferrors caused by human factors in both the early and thelater stages of product development. Also, automation ofdesign gives more spare time to designers which can beused for the solution of more complex problems. This leadsto rise in the quality of an end product.
Keywords—ontology approach, Design Cockpit, Robot-Designer, automated design
I. INTRODUCTION
Design is a complex decision making process in conditionsof an indeterminateness. At present, when designing complexsystems, it is necessary to understand that they represent anetwork of objects interacting in various fields of knowledge(for example, in mechanics, electrics, etc.). A successful un-derstanding of their interaction is determined by an all-aroundtheoretical understanding of this system and an understandingof the process of its design. Traditional approaches today areassociated with the active participation of people at all stages
of design, despite the fact that there are already many standardsolutions that need to be automated. The software is used asa tool, and not as an intellectual assistant to the designer, asfor the most part, there is no accumulated knowledge base in aparticular subject area. The difference between the consideredapproaches in a much deeper formalization of processes basedon ontologies of subject areas. Automation should be imple-mented to routine, repetitive processes. Finding and correctingerrors made during these processes can significantly increasethe design time. Thus, the automation of the entire product lifecycle helps engineers to solve more important and complextasks, which will certainly lead to a reduction in design time andimprove product quality. In the future, success in the industrialsector will be determined by the use of various informationtechnologies to support the design and production processes,but now we can face several problems, which are:
• Separated data sources, which is caused by a data struc-ture that represents fragmented “islands”, each of which isa certain knowledge of the subject, but which are difficultto combine together.
• The inconsistency of the process arises as a result of thefact that not all processes are established. Each companyhas its own vision of the process.
• Data exchange as a written documents implies that partof the design time is spent not on the design itself, buton reading documents that describe the previous work.
• Creating and updating models manually, as in most cases,to research an object, it is necessary to create differentmodels for different purposes.
II. DESIGN COCKPIT 43®
A. Formalization of the design processThe overall design sequence is presented in Figure 1. The
red arrows show that the transition to the next stage is possibleonly with the successful completion of the current one, and theyellow arrows show that if it is impossible to find a solutionat this stage, you should return to the earlier design stages andmake changes to them. It should be noted that the zero stageis expedient only if it is automated.
Due to the antagonistic design principles “form followsfunction” and “function follows form”, geometry and physicsare potentially both at the same time a requirement or result ofa design process, depending on the design context. In graph-based design languages the representation of geometry andphysics is called abstract, since both geometry and physics(i.e. loads, boundary conditions, materials) are represented
165
Figure 1. Design sequence.
independently of any vendor specific, proprietary data formator tool in the Unified Modeling Language (UML).
At the conceptual design stage, geometry is often presentedas parameters and fixtures. At stage 3, all necessary calculationsare performed. At stage 4, the interaction of components isdetected, after which, at stage 5, an analysis of failures isperformed by methods such as FTA and FMEA. Next, thelayout of the components is drawn up, and the pipes and cablesare wired. Stage 8 is possible using the design languages usedin the Design Cockpit 43® software [1].
B. Description of design languageA design language allows for a holistic description of
engineering tasks and is words that form some vocabulary andrules that make up some grammar.
The rules encode model transformations, create instances,and work with separate vocabulary blocks, which, in turn, areencoded in the extended UML instance diagram. The set ofall rules is called the production system, which is encodedin the UML action diagram. These three parts form a graph-based design language and must be created manually by oneor more people as an advance contribution to the engineeringdesign process. When a production system is executed by aso-called design compiler (for example, Design Cockpit 43®),the design result automatically created and stored in the centralmodel is called a graph, that is, a complete digital model of thesystem containing all parts, connections and parameters. Fromthis central data model, all other necessary system models aregenerated automatically. For example, you can generate a CADmodel or a wiring model.
Here the design language means that all valid sentences inthe grammar (that is, all regular combinations of words) arevalid in the design of the system. The term “graph-based”means that a single node in a graph serves as an abstract place-holder for a design knowledge element (i.e. Concepts, values,physical component or their totality), and graph edges express(potentially N-dimensional and interdisciplinary) links betweendifferent nodes (i.e. different parts of design knowledge). Onthe example of a car: “Words are a car, wheels, a door.” Rules:(A) if there is nothing, create a car, (B) if there is a car, createits wheels, (C) if there is a car wheel, attach the door. Thenproduction takes the following form: (A) once, (B) once, (C)four times. The resulting graphic scheme has one auto-mobilewith wheels and 4 doors.
C. Aviation approachDesign languages is applicable in a small scope, e.g. the
automation of a specific task in an established design process.
Figure 2 shows the automation of parts of the design processof an aircraft cabin, with the generation of fioorplan, 3D-modeland wire harness model. Automated design of an aircraft cabinincluding routing. From left to right: a set of requirements(not shown) drives the generation of the cabin layout, manualintervention is possible, e.g. to move the door to another frameif desired. According to the layout pre-constructed geometry isplaced to generate a CAD-model. This then allows to calculatethe maximum installation space for cables, the routing space. Inthis routing space network components, e.g. electronic boxes,are placed as start and end points for the routing algorithm. Aslast step the Design Cockpit 43’s routing algorithm generatethe cables and the electrical wire harness.
Figure 2. Automation of parts of the design process.
The composition of the cabin harness is directly coupledwith the cabin layout. For each seating row there is an overheadpanel which includes reading lamps, buttons to call for flightattendees etc. additionally there may be an in-flight entertain-ment system with screens and audio jacks. These functionsare driven by a small electronic box installed on top of theceiling panel of the seating row. Depending on the chosenelectrical architecture these small boxes are in turn connectedto bigger management nodes. The number of managed boxesdetermines the size of those management nodes - a point foroptimization, a few big or many small ones. Thus the numberand positions of the seats determines directly the position andnumber of the electronic components which in turn define theharness length and architecture. The automated design processbegins with the generation of a cabin layout from a set ofrequirements, e.g. evacuation times, seat distances, passengercapacity, and so on. The cabin layout can be visualized withan automatically generated fioorplan. Once the layout is fixedthe CAD-model of the cabin interior is generated by loadingpre-defined 3D-geometry, e.g. seats or overhead bins, at therespective coordinates. Now the routing space can be extracted,this is the maximum possible volume where cables could beplaced. Components of the electrical network, e.g. electronicboxes, are placed inside the routing space as start and endpointsfor the routing algorithm. In a last step, the Design Cockpit 43’srouting algorithm generates the wire harness of the aircraftcabin. With this automated process in place, quick variationstudies in the form of “What happens if...” are possible, e.g.the door is moved by one (fraction of a) frame or the lavatoryto passenger ratio is changed [1].
III. ROBOT-DESIGNER
A. Knowledge formalizationThe task of simulating the activities of a project involves
not only describing the project operations themselves and theprocedures performed, but also translating them into a formalaction language. Thus, the initial phase of any pre-project studyinvolves the study of the experience and properties of alreadycreated artifacts; This may affect the parameters of the futureproject.
In the conceptual design of the aircraft, such studies arecarried out on the basis of the study of trends, the constructionof statistical models. To formalize this process, a database
166
of airplanes, engines, aerodynamic profiles, avionics, etc. iscreated and updated. Based on the experience, the most de-manded and influencing dependencies of the desired parametersthat “help” the designer in assessing and making decisions areinfluenced. All these actions are described, logged and furtherformalized.
Until recently, difficultly formalized tasks, for example, likethe automatic construction of a grid of finite elements on ageometric model, are more subject to the software packagesthat are being created.
Having achieved in a number of areas the formalization ofknowledge through the identified laws, physical and heuristicpatterns, for the further translation of knowledge to a computer,the task of semantic data consistency came to the fore.
An ontology in the form of a thesaurus explicitly providesthe information necessary to understand the term in a termino-logical system, which is the complete semantic environment ofeach term connected by a semantic network. Small part of thisdescription presented in figure 3.
Figure 3. Example of wing parameters description.
The ontological approach to the study and research of thedomain makes it possible to view the entire set of words, whichcan be used to describe the topic of interest, while reviewing thesemantic environment in which it is interested. The terminologybase and methods for its expansion may change both duringthe creation and use of the thesaurus, therefore, to determineinformation materials on the domain, the relevance of sourcesand scenarios for the use of ontology are taken into account.
B. Design ScenarioAircraft pre-design is chosen as the Subject Area for the
Designer Robot. On the one hand, this is a field of activity thathas always required creative solutions, on the other - it is fairlywell formalized. The result of the work of the Robot Designeris the model of the aircraft. The preliminary design stage ofthe aircraft includes the development of a general concept ofthe designed object, the compilation of models of the objectelements, the preparation of a feasibility study, the formationof a design task.
The description of the object includes its design scheme,approximate estimation of mass, overall dimensions and energyconsumption.
The Robot-Designer is a computer with peripheral devices,toolkits that include machine description languages, a databasemanagement system (DBMS), CAD systems, ontology editors,and a knowledge base, as a combination of thesaurus, database,rules and procedures, with design scenarios. The enlarged blockdiagram of the Robot-Designer is presented in Figure 4.
The Robot-Designer can work in the automatic mode or inthe mode of the intellectual assistant of the human designer,
Figure 4. Structure of Robot-Designer.
and the degree of human participation in the design is notconstant and depends on the desire of the user. In other words,for each user, in advance or dynamically in the process ofwork, a communication script is created, including the degreeof automation of the design process, the choice of preferredinput / output devices, the need to perform certain stage ofdesign. Developed by the Robot-Designer, due to hardware andsoftware limitations, it is not able to independently synthesizefundamentally new versions of the design-power circuit, so thesystem uses those versions of the design patterns that werepreviously described in it.
The Robot-Designer allows analyzing a number of variantsof the aircraft’s layouts and configurations, either independentlyor, if necessary, on the basis of a dialogue with the designer,select the option that best meets the specified technical require-ments.
Developed Robot-Designer, due to hardware and softwarelimitations, it is not able to independently synthesize fundamen-tally new versions of the design-power circuit, so the systemuses those versions of the design patterns that were previouslydescribed in it.
The Robot-Designer allows analyzing a number of variantsof the aircraft’s layouts and configurations, either independentlyor, if necessary, on the basis of a dialogue with the designer,select the option that best meets the specified technical require-ments.
C. Geometry model
In work the method is used, allowing to create geometricalmodels of the plane in an automatic mode with the help ofparametric modeling technology. Any design process as a setof methods of analysis and synthesis includes a set of rulesand methods. They can be generalized and implemented bysoftware into some kind of convolution, conventionally called“parametric template”. When using templates, the designer onlyneeds to enter input data. At the output, whole constructionsare built according to the knowledge and algorithms for solvingproblems laid down in the template. Templates enable the oncecreated algorithms to be re-applied to other constructions, whileobtaining a new result.
Figures 5 and 6 show the created 3D geometric models ofthe aircraft, Figure 7 shows the resulting structural-power andvolumetric layout of the aircraft. These models can be used asa basis for subsequent engineering analysis in CAE-systems, aswell as for physical experiments on a solid-state model obtainedon a 3D printer.
167
Figure 5 shows the constructed 3D geometric model ofthe aircraft, Figure 6 shows the resulting structural-power andvolumetric layout of the aircraft. These models can be used asa basis for subsequent engineering analysis in CAE systems, aswell as for physical experiments on a solid-state model obtainedon a 3D printer [2].
Figure 5. 3D model of a light regional aircraft.
Figure 6. 3D layout model.
IV. CONCLUSION
The considered methods and approaches help to work withcomplex systems, which is undoubtedly relevant at the moment,since day by day products become more complex and requirea huge amount of knowledge in various fields of science forsuccessful development of their concept, structure and furtherproduction. Ontology approach makes automation of desingpossible.
Thy next aims are: further formalization of template deci-sions, research of the decision-making mechanism, implemen-tation of automated procedures for evaluating design decisionoptions, development of a friendly interface based on usability.
REFERENCES
[1] J. Schmidt, "Total Engineering Automation," IILS white paper.[2] N.M. Borgest, S.A. Vlasov, Al.A. Gromov, An.A. Gromov, M.D.
Korovin, D.V. Shustova, Robot-Designer: on the road to reality,Ontology of designing, 2015; 5(4): 429–447.
[3] : S. Vogel, S. Rudolph, Complex System Design with DesignLanguages: Method, Applications and Design Principles. Ontol-ogy of designing. 2018; 8(3): 323-346. - DOI: 10.18287/2223-9537-2018-8-3-323-346.
[4] N.M. Borgest, A.A. Gromov, A.A. Gromov, R.H. Moreno,M.D. Korovin, D.V. Shustova, S.A. Odintsova, Y.E. Knyazihina,“ROBOT-designer: fantasy and reality, Ontology of designing,2012; 4(6): pp.73–94.
[5] S. Rudolph, “Know-How Reuse in the Conceptual Design Phaseof Complex Engineering Products – Or: “Are you still con-structing manually or do you already generate automatically?”Invited Paper, Conference Proceedings Integrated Design andManufacture in Mechanical Engineering 2006 (IDMME 2006),May 17-19th, Grenoble, France.
[6] Rolf Alber and Stephan Rudolph,“"43" - A Generic Approach forEngineering Design Grammars,” AAAI Technical Report SS-03-02.
[7] Thomas Kormeier and Stephan Rudolph, “ON SELF-SIMILARITY AS A DESIGN PARADIGM,” Proceedings ofIDETC/CIE 2005 ASME 2005 International Design EngineeringTechnical Conferences & Computers and Information inEngineering Conference September 24-28, 2005, Long Beach,California, USA.
[8] S. Rudolph,P. Arnold, M. Eheim, S. Hess, M. Motzer, M. Riesten-patt genannt Richter, J. Schmidt, R. Weil,”Design languages formulti-disciplinary architectural synthesis and analysis of complexsystems in the context of an aircraft cabin," CEAS Conference,Toulouse, November 25-27, 2014.
ОНТОЛОГИЧЕСКИЙ ПОДХОД КПРОЕКТИРОВАНИЮ СЛОЖНЫХ
АВТОМАТИЗИРОВАННЫХ СИСТЕМ НАПРИМЕРЕ САМОЛЕТА
Малочкина А. В., Боргест Н. М.В данной статье рассматривается процесс автоматизациипроектирования сложных систем. Приведен обзор существу-ющих подходов, основанных на использовании онтологий:Робот-проектант, созданный в Самарском университете, иDesign Cockpit 43®, компилятор, использующий языки про-ектирования, созданный в Штутгартском университете. Дляробота-проектанта рассматривается формализация знаний,процедур проектирования и операций в выбранной областизнаний, а также семантические и математические модели.Особое место в создании робота-проектанта занимает ин-терфейс, который предоставляет разработчику информациюдля принятия окончательного решения и, при необходимо-сти, «объясняет» необходимость выбора конкретного ре-шения. Для Design Cockpit 43® дается описание языковпроектирования: лексика, правила, грамматика, структураинформации в языке.]
Приведено описание и способы реализации онтологии.Задача - рассмотреть уже существующие методы автома-тизации проектирования самолетов. Это влияет на фор-мализацию процесса проектирования как такового и, со-ответственно, этапов, на которых наиболее целесообразноавтоматизировать деятельность человека. Примеры пока-зывают, как применить это на практике в современномпроизводстве. Актуальность статьи обоснована растущиминтересом к автоматизации проектирования и производства,поскольку она сокращает время проектирования и уменьша-ет количество ошибок, вызванных человеческим факторомкак на ранних, так и на более поздних стадиях разработкипродукта. Кроме того, автоматизация проектирования даетбольше свободного времени констукторам, которые могутиспользовать его для решения более сложных задач, чтоприводит к повышению качества конечного продукта.
Received 11.01.19
168
Cognitive map as representationof knowledge structure
Svetlana ZbrishchakFinancial University under the Government of the Russian Federation
Moscow, [email protected]
Abstract—Tacit knowledge management requires a spe-cial approach because of the complexity of verbalization,explanation and formalization. The purpose of the articleis to substantiate the approach to the tacit knowledge man-agement, which allows integrating concepts from differentfields of knowledge on the basis of systems methodology.Problems and tasks of tacit knowledge management areconsidered in the context of social interaction of individuals.This raises the problem of mutual understanding andcommunication barriers. The challenge of understandingdepends on the quality of knowledge, the way knowledge ispresented and the coherence of knowledge among them-selves. An interdisciplinary approach based on systemsmethodology and soft systems thinking is proposed. Thestructuring of knowledge occurs through the constructionof a collective cognitive map, which is a conceptual systemsmodel of knowledge of a group of individuals. The elementsof the model are ideas, assumptions, judgments, opinions ofindividuals, and the process of construction is consideredas a way of organizing social interaction, which is based onthe formation of a shared understanding. Priority directionsof further research include the development of methods ofanalysis, verification and evaluation of the credibility ofmodels based on collective cognitive maps.
I. INTRODUCTION
Problem solving in human activity is associated withthe organization of various types of knowledge. The is-sues of supporting human intellectual activity are studiedin various fields (cognitive psychology, knowledge engi-neering, knowledge management, management sciences,system analysis, etc.). However, there is still a shortageof relatively simple and convenient tools for managingthem, especially when solving real life problems. Forquite a long time, the field of knowledge managementhas been associated with IT technologies, whose devel-opment has been promising over the past few decades. Itseemed that the formalization of organizational knowl-edge, the creation of corporate knowledge bases, portalscan effectively manage knowledge and provide supportfor intellectual activity. However, in solving real lifeproblems and tasks, it was found that knowledge man-agement is not only and not so much the creation ofknowledge bases or portals. It turned out that the mostvaluable knowledge needed for supporting intellectualactivity is tacit knowledge. Tacit knowledge is individualin nature and largely depends on the cognitive character-
istics of individual. In addition, this kind of knowledge isdifficult to formalize, and therefore is almost impossibleto spread it through “uploading” to corporate knowledgebase. Obviously, management of tacit knowledge requiresa different approach to knowledge management [1].
The term “knowledge” and, as commonly used with it“information”, are polysemantic. Depending on applica-tion they are interpreted differently, and are often usedon an intuitive level. Studies in psychology show thata person actively processes information, creating certainmeaningful conceptual structures, which are consideredas knowledge. Conceptual structures are considered asa “special level of cognitive organization”, in which anindividual version of the world picture is constructed,that determines the activity in typical conditions. [2]
II. ON SOME PROBLEMS AND TASKS OF TACITKNOWLEDGE MANAGEMENT
Traditionally, the “bottleneck” in knowledge man-agement is the extraction, representation of knowledgeand conceptual analysis (or knowledge structuring) [3].Along with this, the spread and exchange of knowledgeis also difficult. Spread of tacit knowledge involves theexchange of ideas, experiences; an explanation of thelogic that was used to solve problems or tasks in thepast in order to help other people solving other problemsand tasks in the present or future. From this perspective,the exchange and spread of knowledge is based oncommunication between individuals and is consideredas social interaction. In this case, in the social interac-tion of a group of individuals, the problem of mutualunderstanding arises, which depends on the knowledge,their quality, ways of presentation and coherence ofknowledge among themselves [4]. The communicationbarriers arising from this are due not only to differencesin knowledge, but also to the subjectivity of judgmentsand assessments that reflect the cognitive characteristicsof a person: perception, interpretation, understanding ofthe surrounding world. In the areas related to the de-velopment of new products, communication barriers aredefined as knowledge boundaries: syntactic, semantic,and pragmatic. They are manifested through differencesin the knowledge, experience, views and interests ofgroup members [5].
169
Based on this the research task is not only to iden-tify and conceptualize tacit knowledge of an individual,but also to organize the processes of spread and shareknowledge between individuals. This leads to the taskof organizing the processes of communication and socialinteraction in a group of individuals.
A. Systems methodology and tacit knowledge manage-ment
The tasks of knowledge elicitation and conceptu-alization are usually solved on the basis of systemsmethodology. The wide application of systems approachin the second half of the 20th century for solving variousproblems in socio-economic systems, where the role ofthe active elements of the system is played by people,whose individual and collective behavior determines theessential aspects of the behavior of the system as a whole,revealed some limitations of its application. The tradi-tional systems approach, called later “hard” or “hard sys-tems thinking”, seeks to bring scientific rigor to problemsolving and aims to produce objective results that are freeof the personal aspect. The recognition of the significanceof the “human factor” and the associated risks has ledto the creation and development of “soft” systems ap-proach or “soft systems thinking”. Soft systems thinkingconsiders a person and his perception, beliefs, values andinterests as basic components of system. The main task,which is solved with the help of soft systems thinking, isto identify world views and system of assessments thatpeople use to understand and construct reality. From thisperspective soft systems thinking has been designed toovercome the shortcomings of hard systems thinking.
At present among the well-developed and widely ap-plicable soft system methodologies can be distinguished"Soft Systems Methodology" (SSM), developed by P.Checkland, "Strategic Options Development and Anal-ysis" (SODA) by C. Eden and "Strategic Choice Ap-proach" (SCA) by J. Friend.
Thus, the second research issues consists in findingadequate forms of system representation, the componentsof which are perception, beliefs, values and interests ofactors. More broadly we can talk about adequate forms ofelicitation and representation of the knowledge structuresof an individual and/or a group of individuals. Fromthis it follows that the system methodology should becomplemented with concepts of cognitive science.
B. The aid of cognitive science
In cognitive science knowledge structures are usuallyconsidered as mental representations. The concept of"mental representation" refers to the number of key con-cepts of cognitive science and is defined as "...the actualmental image of a particular event (that is, the subjectiveform of "vision" of what is happening). ... mental repre-sentations are an operational form of mental experience,they change as the situation and intellectual efforts of the
subject change, being a specialized and detailed mentalpicture of the event" [6, p. 98]. The recognition ofthe presence of representation is the recognition of theexistence of an "internal" reality, i.e. the representationof reality in the consciousness of the individual. Fea-tures of representation formation determine the natureof intellectual activity. Mental representation is a "built"reality in certain conditions and for certain purposes.At the present stage, representation is considered notonly as a form of knowledge storage, but also as atool for applying knowledge to certain events, objectsof reality. The role of the representation of informationis most clearly manifested in the processes of solvingproblems and consists in creating an adequate conceptualunderstanding of the problem situation, which, in turn,serves as foundation for integrating and transforminginformation.
III. COGNITIVE MAP AS TOOL FORCONCEPTUALIZATION KNOWLEDGE
To spread knowledge it is necessary not only toidentify them, but also to represent (visualize) them inthe most convenient form for human perception. To date,various methods of knowledge visualization have beencreated and are widely used (e.g. [7 - 9]). Along with thiseffective method of visualization are different types ofcognitive maps. The definition of the concept “cognitivemap” is rather vague [10, 11] and depending on the fieldof study or application is used for distinguishing betweenmental representation, which exists only in mind, and itsexternal representation; or, according to R. Axelrod [12]map is not “cognitive map”, but “map of cognition”. C.Eden [13] uses this concept in a completely differentway: a cognitive map is not a “map of cognition”, but a“map created to help cognition”. In the field of artificialintelligence a similar to R. Axelrod approach is used tomap the knowledge of experts, but combined with fuzzylogic to build fuzzy cognitive maps.
It is obvious that the term “cognitive map” has suchan intuitive application that new researchers appear withnew ideas or mapping techniques for completely new anddifferent purposes.
The cognitive mapping technique is based on the po-sition of existence of cognitive functions of informationprocessing, which directly affect human behavior andactions. Cognitive mapping is a technique of graphicrepresentation of various individual views on the issuesunder consideration. In general, mapping techniques canbe divided into two large classes:
1) to represent the cognitive processes of the individ-ual;
2) to represent cognitive processes at the group level.In order to solve the problem of knowledge spread
it is necessary to aggregate external representations ofcognitive structures and processes of individuals. In this
170
case, we are not talking about a simple aggregation ofindividual knowledge structures, but about creating acollective knowledge structure and its visualization.
A. The aid of conceptual modeling
Another aspect of the “bottleneck” in knowledge man-agement is knowledge structuring. The solution to thisproblem is found with the help of conceptual modeling.Conceptual modeling can be defined as a way of decidingwhat to include in the model and what to exclude fromthe model [14]. Unfortunately, this type of modelingis not well understood. The main reason for this isprobably due to the fact that conceptual modeling ismore an “art” than a “science”. Therefore, it is difficultto define methods and procedures, and the skills ofconceptual modeling are acquired for the most part onlythrough experience in solving practical problems. Amongthe key aspects of conceptual modeling, the followingcan be highlighted: iterativeness, independence from thesoftware or development environment used, significanceof the positions and points of view of both the modeldeveloper and the client.
Thus, conceptual models are a visualization tool andhave been designed for formation of primary knowledgeand their holistic perception. These models are a con-venient tool for structuring and representing knowledge,especially in the early stages of the study of the subjectarea, and allow to describe it in the form of conceptsand relationships between them. Conceptual models areused not only for the representation and integrationknowledge, but also for training, knowledge transferringand share.
Since the spread of knowledge is communication andsocial interaction, the methods of group modeling weredeveloped for supporting these processes. Group modelbuilding methods allows to coordinate and collect scat-tered knowledge of the participants in the system model.The model is considered as a form of representation(visualization) different points of view, judgments and as-sumptions of group members, and the process of buildinga model is a way of organizing social interaction. At thesame time, the identified primary ideas (knowledge) ofthe participants in the process of refinement, coordinationare transformed in such a way that a new integratedknowledge is created, which none of the members ofthe group had previously possessed.
Thus, the task of spread (also transferring and share)knowledge is reduced to the construction of a collectiveconceptual system model of the issue under considera-tion.
B. Oval Mapping Technique for building collective cog-nitive map
At the stage of knowledge conceptualization the OvalMapping Technique (SODA methodology) seems to be
quite convenient. The method is based on the construc-tion of a collective causal map in the form of a directedgraph, the nodes of which are related causality or influ-ence concepts (expressing ideas, assumptions, judgments,opinions). [15] The process of building includes the stepsof concepts elicitation, clarification, coordination, struc-turing. The created model has a hierarchical structure,which greatly facilitates reading and analysis of map.
However, in situations characterized by novelty anduncertainty building of causal models is very difficultbecause of cognitive limitations of individuals – inconditions of uncertainty it is extremely difficult for anindividual to build causal relationship. Under these con-ditions, at the initial stage, it is proposed to build maps ofinfluence - to determine only the impact of concepts oneach other, without specifying the type and strength ofinfluence. Further, to the extent of clarifying the languageand meaning of concepts and their coordination (this mayrequire to elicit additional concepts) it becomes possibleto determine the type of links between them. [16]
From the perspective of creation collective knowledge,the processes of refinement, coordination and accommo-dation are of interest. At the heart of the transformationof individual knowledge and their aggregation in the formof a model is the formation of a shared meaning andshared understanding – notions that are the subject ofresearch of social psychology.
A model built using group model building techniquesvisualizes a holistic shared view of a group of individualsabout the issue under consideration. For creating a com-mon image of a system as opposed to the individual oneit is necessary to form shared understanding of elementsand their interrelation in the model. The shared under-standing can be defined as “the overlap of understandingand concepts among group members” [17, p. 36]. Forcollaborative modelling shared understanding is seen as“the extent to which specific knowledge among groupmembers of concepts representing system elements andtheir relations overlaps” [18, p. 249]. For creating overlapof knowledge the participant not only need to exchangeinformation about the elements of the model and theirinterrelation but also to form a shared meaning of theseelements and their interrelation. The formation of sharedmeaning is usually viewed from the point of view ofsensemaking, understood as “the ongoing retrospectivedevelopment of plausible images that rationalize whatpeople are doing” [19, p. 409].
Although the mechanisms of formation of sharedmeaning and shared understanding are not sufficientlystudied, poorly understood, nevertheless, reliance onthem can partly solve the problem of verifying theadequacy and credibility conceptual model.
IV. CONCLUSION
The author’s application of cognitive mapping meth-ods for solving real life problems [20] has shown that
171
in addition to the mastery of conceptual modeling, isrequired knowledge from various fields (system analysis,cognitive science, etc.) both for the practical appli-cation of methods based on cognitive maps, and forthe theoretical justification of the building technologyand obtained results. The further development of thisarea requires not even an interdisciplinary approach,but a transdisciplinary one, which may create a com-mon conceptual space for different areas of knowledge,thereby, if knowledge boundaries are not removed, thensignificantly reducing them. Among the priorities forfurther research can also be identified the developmentof methods of analysis, verification and evaluation of thecredibility of models based on collective cognitive maps.
REFERENCES
[1] McDermott R. Knowing in community. IHRIM, 2000, no. 19. pp.19-26.
[2] Velichkovskij B.M. Kognitivnaya nauka: Osnovy psikhologii poz-naniya [Cognitive science: Fundamentals of cognition psychol-ogy], Moscow: Smysl, 2006, (in Russian).
[3] Gavrilova T.A., Leshcheva I.A., Strahovich E.V. Ob ispol’zovaniivizual’nyh konceptual’nyh modelej v prepodavanii [On theuse of visual conceptual models in teaching]. Vestnik Sankt-Peterburgskogo universiteta [Bulletin of St. Petersburg University],2011, no. 4, pp. 124-150 (in Russian).
[4] Abramova N.A. Refleksivnyj podhod i problema vzaimoponi-maniya [Reflexive approach and the problem of mutual under-standing]. Chelovecheskij faktor v upravlenii [Human factor inmanagement], Moscow: KomKniga, 2006, pp. 52-82 (in Russian).
[5] Carlile R.P. A pragmatic view of knowledge and boundaries:Boundary objects in new product development. OrganizationScience, 2002, no. 1, pp. 442-455.
[6] Holodnaya M.A. Psihologiya intellekta: Paradoksy issledovaniya[The psychology of intelligence: Paradoxes of research], St. Pe-tersburg: Piter, 2002. 272p. (in Russian).
[7] Gavrilova T.A., Gulyakina N.A. Vizual’nye metody raboty soznaniyami: popytka obzora [Visual methods of working withknowledge: an attempt to review]. Iskusstvennyj intellekt i prinyatiereshenij [Artificial Intelligence and Decision Making], 2008, no.1, pp. 15-21 (in Russian).
[8] Gavrilova T.A. et al. About one method of classification of visualmodels [About one method of classification of visual models].Biznes-informatika [ Business Informatics], 2013, no. 4 (26), pp.21–34 (in Russian).
[9] Gavrilova T.A., Alsuf’ev A.I., Grinberg E..YA. Vizualizaciyaznanij: kritika Sent-Gallenskoj shkoly i analiz sovremennyh tren-dov [Knowledge visualization: criticism of the St. Gallen schooland analysis of modern trends]. Biznes-informatika [Business In-formatics], 2017, no. 3 (41), pp. 7–9 (in Russian).
[10] Doyle J.K., Ford D.N. Mental models concepts revisited: someclarifications and a reply to Lane System Dynamics Review: TheJournal of the System Dynamics Society, 1999, v. 15, no. 4, pp.411-415.
[11] Kitchin R.M. Cognitive maps: What are they and why studythem? Journal of environmental psychology, 1994, v. 14, no. 1,pp. 1-19.
[12] Eden C. Analyzing cognitive maps to help structure issues orproblems. European Journal of Operational Research, 2004, v.159, no. 3, pp. 673-686.
[13] Axelrod R.Structure of decision: The cognitive maps of politicalelites, Princeton, NJ: Princeton University Press, 1976. 400 p.
[14] Robinson S. et al. (ed.). Conceptual modeling for discrete-eventsimulation, CRC Press, 2011. 490 p.
[15] Ackermann F., Eden C. Strategic options development and anal-ysis.Systems approaches to managing change: A practical guide,Springer, London, 2010. pp. 135-190.
[16] Zbrishchak S.G. Sistemno–kognitivnyj podhod k organizaciisovmestnoj deyatel’nosti gruppy zainteresovannyh storon [Sys-tem–cognitive approach to the organization of joint activities ofa group of stakeholders]. Ekonomika i upravlenie: problemy, resh-eniya [Economics and Management: problems, solutions], 2017,v. 3(66), no. 6, pp. 155-158 (in Russian).
[17] Mulder I., Swaak J., Kessels J. Assessing learning and sharedunderstanding in technology-mediated interaction. EducationalTechnology and Society, 2002, no. 5(1), pp.35-47.
[18] Renger M., Kolfschoten G.L., de Vreede G.J. Challenges incollaborative modelling: a literature review and research agenda.International Journal of Simulation and Process Modelling, 2008,no. 4(3-4), pp. 248-263.
[19] Weick K.E., Sutcliffe K.M., Obstfeld D. Organizing and theprocess of sensemaking. Organization science, 2005/ no. 16(4),pp. 409-421.
[20] Zbrishchak S.G. Reshenie problemnyh situacij v menedzhmentena osnove kollektivnyh kognitivnyh kart [Solving problem situa-tions in management on the basis of collective cognitive maps].Ekonomika i upravlenie: problemy, resheniya [Economics andManagement: problems, solutions], 2017, v. 4(63), no. 3, pp. 235-245 (in Russian).
КОГНИТИВНАЯ КАРТА КАКРЕПРЕЗЕНТАЦИЯ СТРУКТУР ЗНАНИЙ
Збрищак С.Г.
Управление неявными знаниями требует специаль-ного подхода в силу сложности вербализации, объяс-нения и формализации. Цель статьи состоит в обос-новании подхода к управлению неявными знаниями,позволяющий интегрировать положения из различ-ных областей знаний на фундаменте системной ме-тодологии. Проблемы и задачи управления неявнымизнаниями рассматриваются в контексте социальноговзаимодействия индивидов. При этом возникают про-блема взаимопонимания и коммуникативные барьеры.Проблема достижения взаимопонимания зависит откачества знаний, способов представления и согласо-ванности знаний между собой. Предложен междис-циплинарный подход на основе построения коллек-тивной когнитивной карты. Методологической осно-вой служит системная методология в части мягкогонаправления, а структурирование знаний происходитпосредством построения коллективной когнитивнойкарты, которая представляет собой концептуальнуюсистемную модель знаний группы индивидов об ис-следуемой предметной области. Элементами моделислужат идеи, предположения, суждения, мнения ин-дивидов, а процесс построения рассматривается какспособ организации социального взаимодействия, воснове которого лежитформирование совместного по-нимания посредством совместного придания смысла.Приоритетные направления дальнейших исследованийвключают задачи развития методов анализа, проверкии оценки достоверности моделей на основе коллектив-ных когнитивных карт.
Received 30.12.18
172
Implementation of an adaptive model of inputand editing information based on XSLTtransformations for heterogeneous dataAigul Mukhitova
Novosibirsk State UniversityNovosibirsk, Russian Federation
Oleg L. ZhizhimovInstitute of Computational Technologies
Siberian Branch of the Russian Academy of Sciences (ICT SB RAS)Novosibirsk, Russian Federation
Abstract—Adaptability, the ability of interfaces to adaptto the structure and functionality of information sourcesis one of the main features of the information system’s in-telligence. Development of adaptive graphic web interfacesbased on XML-technologies allows to visualize any struc-ture of the XML-format file for further manipulation ofdata input and editing. The paper deals with the technologyof constructing an adaptive graphical administrative WEB-interface for data input and editing in a heterogeneousinformation environment based on the use of XSD dataschema definitions with the use of XSLT transformations.An example of implementation of the adaptive model ofinput and editing of information in the form of the createdprototype of the XML-records editor is given. This editor, inthe client-server architecture, on the server side generatesan empty editing form by converting the modified XSDstructure by XSLT method and then provides the client-side ready HTML-form with all the necessary tools (javascripts) for correct input and/or editing of heterogeneousdata.
Keywords—adaptive graphical user and administrativeweb-interfaces, integration of heterogeneous data, datarepresentation, new data analysis methods, XML, XSD,XSLT-transformations, XML-editor.
I. INTRODUCTION
To create responsive web-based administrative inter-faces for data input and editing, the XML format isthe most appropriate format from all available structuredformats extracted from the relevant information sources.Namely this format allows the final user to work with alarge number of heterogeneous data from many hetero-geneous sources, as well as it has a good data type andallows the definition of its structure.
XML is more dynamic and allows to easily generatenew data schemas and rules of transition between them,which are formulated in the same language (XSLT-transformation), and is supported by a large numberof software manufacturers. The XML standard includessupport for Unicode encodings, which makes it possibleto use several different languages in the same applicationat the same time. XML provides for the transfer ofdata, such as graphics or audio/video, without which the
information environment is unthinkable. A particularlyimportant advantage of XML is its integration with theweb environment, as well as platform independence. Allthis makes XML a de facto standard for data exchange.
It became necessary to create fundamentally new high-level applications based on the integration of informationtechnologies and ensuring the integration of heteroge-neous information resources. This direction is activelydeveloping in many scientific centers of different coun-tries and is associated with the creation of informationsystems for XML-messages exchange, functioning in theWeb environment.
There are a number of works that have been analyz-ing the problem of adaptive web application interfaces,giving possibility to input and output information in xmlformat. In work [1] is represented model and algorithmfor constructing a syntax-directed editor of xml descrip-tions. The main purpose of intelligent XML editors is toprovide a high-level user interface. Such interface mustensure full information on possible user actions at eachpoint of dialogue and on appropriate XML tags. The edi-tor can be built around a logical processor — realizationof a special abstract automaton. The control table of thisautomaton is built using the contents of the DTD fileand based on calculation of the modified Wirth—Weberratios. This approach allows the user to create syntacti-cally correct XML files without knowledge of DTD [1].The author of another paper [2] proposes the conceptof XML documents with a built-in dynamic model. Ageneral structural diagram of this model is presented,and a method for its interpretation is described. Thearchitecture of the developed software tool for creatingand maintaining dynamic XML documents is discussed,and a brief overview of its modules is given. One of thelast works in this direction is the work [3], where dealswith modern and previously proven in practice meansto verify the structure of the documents to the appropri-ate document description scheme.The problem of usingschemes for validating (validating) XML documents in
173
the model of a situationally-oriented database (SODB)with the help of an XML-based document structure.As a result, the ability to work with third-party SODBextends, XML documents stored on thirdparty servicesare downloaded and used in a web application with apreliminary validation check. Thus, the dynamic modelis expanded with specifications for connecting circuitsto monitor the data being downloaded. As a result, theSODB allows not only to process data, but also tocontrol the data downloaded by users or from third-partyRESTful-services. The obtained results are discussedon the example of the XML file of the informationsystem about dissertational councils [3]. We can statethe absence in the open scientific information space ofdomestic and foreign analogues of our proposed adaptivemodel of input and editing information based on XSLTtransformations for heterogeneous data.
A variety of distributed information systems shouldprovide the ability to manage data from heterogeneousinformation sources, i.e. generate administrative and userinterfaces that provide the ability to manage heteroge-neous database (input and editing).
For homogeneous information sources with a fixedrecord structure, the task of co-building and editing datais simple. While for heterogeneous sources of informa-tion (with an arbitrary recording structure) there is aneed to use adaptive technologies for building graphicalinterfaces with the required functionality [4]. XML is auseful tool for structurally describing data, but it is notintended to represent data visually. For this the XMLdata must be converted into another form that is easy forthe user to view and edit through a browser, such as anHTML document.
Such conversions are performed using constructs de-fined by the XSLT language. XSLT transformations areused to present information to the user, and as a meansto convert XML documents to other formats.
XSLT describes the rules for converting the sourcestructure of an XML document into a destination doc-ument (XML, HTML, Text). The final structure canbe modified in contrast to the structure of the originaltree during the construction process, when the elementsof the original tree can be reordered and filtered, aswell as by adding new elements. Each of the specificinformation resources, as a rule, has a rather limitedrange of possible formats and schemes of providingdata and their possible values. However, due to theheterogeneity of data sources, it is necessary to attractadditional information about a particular informationresource when selecting components that regulate dataprocessing of various information resources.
The development of adaptive web-based graphicalinterfaces based on XML technologies allows to visualizeany structure of the XML-format file for the possibilityof further manipulation of data input and editing.
The technology of creation of the adaptive graphicWEB-interface realizing model of input and editing in-formation for heterogeneous data, built-in heterogeneousinformation system will be considered below.
This assumes that the data can be extracted from theappropriate information sources in XML format. As amatter of principle, other structured formats can be used.The system needs to be activated modules, which convertdata into XML format and back, because the XMLformat is described in the most appropriate technologyfor building WEB interfaces to edit data.
II. DESCRIPTION OF TECHNOLOGY
In paper [4] described technology of constructingadaptive user interfaces for controlling the search ofinformation and method of displaying retrieved infor-mation by using the Z39.50 [5] and SRW/SRU [6] onthe basis of services Explain [7-8] in its various mod-ifications. The implementation of these adaptive inter-faces for the ZooSPACE [9-10] platform is also demon-strated. The ZooSPACE complex is based on severalloosely coupled distributed subsystems providing config-uration (ZooSPACE-L), access to resources (ZooSPACE-Z), user and administrative web-interfaces (ZooSPACE-W), statistics collection (ZooSPACE-S) and monitoring(ZooSPACE-M) of the whole system [9]. The imple-mentation of adaptive user and administrative interfacesin the ZooSPACE-W subsystem should minimize useractions for searching, viewing and editing informationfrom heterogeneous sources.
Depending on the technologies used to access infor-mation resources, input information about the functionalproperties of each data source should be obtained.
To present structured XML information, it is im-portant to have an XSD (XML Schema) data schemadescription. XML Schema technology allows you tocheck the correctness of the XML document according tothe described rules and apply code generation tools forvarious web programming tools, which speeds up theapplication development process. In general, the rulesfor XML are formulated in terms of XSD [11-13] andthey present XML structure which can be processed withstandard ways, for example XSLT [14]. The questionabout where the full description of potential structure ofthe derived record can be obtained arises in the process ofextracting record from a particular informational sourcein heterogeneous informational system and presenting therecord in the XML format for editing. The followingoptions are possible [15]:
• the XML record, derived for editing, includes areference on the applied XSD data scheme in theform of URL with Schema Location as an attributein determination of employed namespaces. It isusually contained in the XML record root element.In that case the issue of receiving XSD is solved ina trivial manner;
174
• the XML record derived for editing, includes thenamespace identification (URI), though it does notinclude a reference on the applied XSD data schemein the form of URL. In that situation the informa-tional system should be requested to provide theXSD in the use of namespace identification. Forthe ZooSPACE platform the similar request can beprocessed with Explain service;
• the XML record, derived for editing, does notinclude definitions of namespaces. In this case theinformational system should be requested to providethe XSD (as a default) by the name of informationalresource (database), or by using the XSD, whichbefore corresponded to the scheme requested in aninquiry formation for extracting data.
For all of the above methods for initializing GUI dataconversion are needed:
1) a description of the data schema in the form ofan XML structure in accordance with the rules ofXSD;
2) an XML structure containing the extracted data forediting (not required to create a new record);
3) a description of the styles of formation of elementsof the graphical interface (optional);
4) description of templates for generating graphicalinterface objects in accordance with XSD rulesand XML record editing elements. Under theseconditions, XSLT transformation rules may applyto XSD.
The functional diagram is presented (see Figure 1), asan illustration of work algorithm of the XML recordsprototype adaptive editor in the format of client-server,built in WEB server of the ZooSPACE (ZooSPACE-W)platform.
Figure 1. The functional diagram of XML-editor in the heterogeneoussystem ZooSPACE-W
III. IMPLEMENTATION AND VALIDATION
The XML editor is relevant to an area restricted bya dashed line for server side (see Figure 1). As for theclient part, the beforehand prepared HTML form to inputand/or edit data is provided. In these conditions, the formalready contains the all needed tools (java scripts) forcorrect data input, which includes:
• a script for checking the accuracy of data entry,if there is a relevant pattern in the way of regularexpression in the XSD;
• a script for removing elements, providing that theremoving is possible according to the XSD;
• a script for duplicating elements, the repetition ofwhich is possible according to the XSD;
• a script for hiding-revealing any data elements inthe form of editing.
It should be taken into account that the XSD datascheme definitions can contain references to other XSDdata scheme definitions, which complement definitionsboth in the current namespace (element xsd:include), andin the other namespaces (element xsd:import). Thereforethe initial XSD structure, before being processed bythe XSLT processor requires modifying to register extradefinitions. The editor of the XML records operatingprinciple, in format of client-server built in WEB server,can be described as follows [15]:
1) as for the client part, the beforehand preparedHTML form to input and/or edit data is provided.In these conditions the form already contains theall needed tools (java scripts) for correct data input;
2) generation of editing forms occurs on the serverside with the XSLT method of transformations ofthe modified XSD structure. At the beginning ofthe process an empty editing form is produced(without data). As soon as the XSLT processor hascompleted its action, the empty form is filled withrecord data in XML format.
For generation of empty form of editing (see Figure2) the following rules are performed:
• The frame indicating the identification of datascheme is generated.
• The file of documents (annotation) for data schemeis generated.
• For each specified data element in XSD the follow-ing is generated:
– the frame indicating the element name and itslocation (in the XPath pattern) in the XMLrecord structure;
– the key button of hiding-revealing element in aform of editing;
– the file of documents (annotation), if any, withan indication of a language; the nested elements(for complex);
– the field of entry element definition (for sim-ple);
175
Figure 2. Graphical interface of XML-editor
– names and data entry fields for each of potentialattributes;
– key buttons for deleting (if allowed) or dupli-cating (if allowed) elements.
• The following key buttons are generated:– “Record” – for storage a result of editing;– “Clear” – for regeneration of empty editing
form;– “Close” – for closing editing form without data
storage.The type of data and the placed restrictions are taken
into account in the process of generation of data entryfields. In particular, the field of entry elements and at-tributes are presented with a list of dropdown definitions(see Figure 3) if there is XSD definitions such as:
Figure 3. Graphical XML-editor: data entry fields
<xsd:simpleType name="recordTypeType"><xsd:restriction base="xsd:NMTOKEN"><xsd:enumeration
value="Bibliographic"/><xsd:enumeration
value="Authority"/><xsd:enumeration
value="Holdings"/><xsd:enumeration
value="Classification"/><xsd:enumeration
value="Community"/></xsd:restriction>
</xsd:simpleType>
If the XSD element contains indication for a pattern(RegEx), for example:
<xsd:simpleTypename="indicatorDataType" id="ind.st"><xsd:restriction base="xsd:string"><xsd:whiteSpace value="preserve"/><xsd:pattern value="[\da-z ]1"/>
</xsd:restriction></xsd:simpleType>
In that case, the access to checking function of cor-respondence with a pattern of data entry in the formof editing is generated, that is XSLT code will beperformed:
...<xsl:for-each
select="xsd:simpleType/xsd:restriction/xsd:pattern"><xsl:attribute name="onChange"><xsl:text>e_change(this,
/</xsl:text><xsl:value-of select="@value"/><xsl:text>/);</xsl:text>
</xsl:attribute></xsl:for-each>...
Which in turn generates the forms of elements
<input type="text" onChange="e_change(this,/[\da-z ]1/);".../>
A problem of recursive definitions arises from thedescribed approach in XML formation on the groundof XSD (see Figure 4). Recursiveness may occur in theappliance of references to types and names. A fragmentof a recursive determination is provided in the schemewith the help of the XSD.
<xsd:complexType name="org"><xsd:sequence><xsd:element name="id"
type="int"/><xsd:element name="name"
type="string"/>
176
<xsd:element name="sub-org"type="tns:org"/>
</xsd:sequence></xsd:complexType>
<xsd:element name="region"><xsd:complexType><xsd:sequence>
<xsd:element name="id"type="int"/>
<xsd:elementref="tns:region"/>
</xsd:sequence></xsd:complexType>
</xsd:element>
<xsd:element name="record"><xsd:complexType><xsd:sequence><xsd:element name="id"
type="int"/><xsd:element name="org"
type="tns:org"/><xsd:elementref="tns:region"/>
</xsd:sequence></xsd:complexType>
</xsd:element>
The XML elements with unrestricted length of Xpathare possible:
/record/organization/sub-org/sub-org/sub-org...
/record/region/region/region/region...
Figure 4. Recursion fragment
The attachment number control can be used for elim-inating the endless number of item attachments in gen-eration of graphic interfaces of editing records and forrestricting them in accordance with the current demand.The list of processed elements, XSD (rules), is depictedby editor prototype in the table I.
Table ISUPPORTED XSD ELEMENTS
Element Attributeannotationappinfoattribute name, ref, type, usechoicecomplexContentcomplexType namedocumentationelement name, ref, type, substitutionGroup, maxOc-
curs, minOccursextension basegroup name, ref, maxOccurs, minOccursimport namespace, schemaLocationinclude schemaLocationlist itemTyperestriction baseschema attributeFormDefault, elementFormDefault,
blockDefault, finalDefault, targetNames-pace, version, xmlns
sequence maxOccurs, minOccurssimpleContentsimpleType nameunion memberTypesunique
IV. CONCLUSION
The above described technology for creating adaptivegraphical Web interfaces for data editing is implementedin a prototype editor, which is a server application.The presented approach to the formation of informationediting interfaces for heterogeneous data allows the de-veloped graphic web interfaces to automatically tune intothe structure of one or another information resource. Thecreated prototype of the described adaptive XML editorallows you to process any XML data by converting thesource data of any structure without any modification ofthe program code. In the future, it is planned to increasethe functionality of the editor in terms of expanding thelist of supported XSD elements and supporting JSONformat, due to its popularity. Upon completion of testing,the editor will be integrated into the ZooSPACE-Wsubsystem of the technological platform for the massintegration of distributed heterogeneous data sourcesZooSPACE. Moreover, it can be used by users as anindependent functional system, in the form of an editorfor working with files in XML format.
ACKNOWLEDGMENT
Work is performed within the Integration Project of SBRAS (AAAA-A18-118022190008-8), Project for basicscientific research (AAAA-A17-117120670141-7) andRFBR project No 18-07-01457-a.
REFERENCES
[1] Yu. M. Sherstyuk, " Model and algorithm for constructinga syntax-directed editor of xml descriptions", "Nauchnoe Pri-borostroenie", 2000, vol. 10, no. 4, pp.72-78
177
[2] V. V. Mironov, G. R. Shakirova, "Programmno-instrumentalnoesredstvo dlya sozdaniya i vedeniya dinamicheskikh xml-dokumentov", Vestnik UGATU, 2007, vol. 9, no. 5, pp.54-63.
[3] V. V. Mironov, N. I. Yusupova, A. S. Gusarenko, "Validationof the xml documents in web-applications based on situation-oriented databases (DTD, XML SCHEMA, RELAX NG)", Pro-ceedings of the 5th All-Russian Conference «Information Tech-nologies for Intelligent Decision Making Support», 2017, May16-19, Ufa, Russia, Volume 1, pp.10-14.
[4] O. L. Zhizhimov, "Explain Services on ZooSPACE Platformand Adaptive User Interfaces", CEUR Workshop Proceedings,2015, Vol.1536, pp.30-36. Available at: http://ceur-ws.org/Vol-1536/paper4.pdf.
[5] ANSI/NISO Z39.50-2003. Information Retrieval (Z39.50): Appli-cation Service Definition and Protocol Speciation. NISO Press,Bethesda, Maryland, U.S.A. Nov 2002. ISSN: 1041-5653. ISBN:1-880124-55-6.
[6] SRU-Search/Retrieve via URL. The Library of Congress. Avail-able at: http:// www.loc.gov/standards/sru (accessed 2016, July9).
[7] ZeeRex: The Explainable "Explain" Service. Available at:http://zeerex.z3950.org
[8] SRU-Explain Operation. The Library of Congress. Avail-able at: http://www.loc.gov/standards/sru/explain (accessed 2013,September 6)
[9] O. L. Zhizhimov, A. M. Fedotov, Y. I. Shokhin, "The ZooSPACEplatform-access organization to various distributed resources.Digital libraries: The Russian scientific e-magazine. Vol.17. No2. ISSN 1562-5419.
[10] O.L Zhizhimov, A.A. Lobykin, I.Y. Turchanovskiy, A.A. Pan-shin, S.A. Chudinov, "Computer -assisted acquisition system ofstatistics event information in distributed information system".Vestnik NSU. Ser.: The Information technology, 2013, Vol.11.ISSN 1818-7900. pp.42-52.
[11] XML Schema Part 0: Primer Second Edition: W3C Recommenda-tion. Available at: http://www.w3.org/TR/xmlschema-0 (accessed2004, October 28)
[12] XML Schema Part 1: Structures Second Edition: W3C Rec-ommendation. Available at: http://www.w3.org/TR/xmlschema-1(accessed 2004, October 28)
[13] XML Schema Part 2: Datatypes Second Edition: W3C Rec-ommendation. Available at: http://www.w3.org/TR/xmlschema-2(accessed 2004, October 28)
[14] XSL Transformations (XSLT) Version 2.0: W3C Recommenda-tion. Available at: http://www.w3.org/TR/xslt20 (accessed 2007,January 23)
[15] A. A. Mukhitova, O. L. Zhizhimov, "Adaptive technolo-gies in the context of designing administrative graphic inter-faces for heterogeneous information systems of inputting andediting data". XVI Russian conference of “The distributedinformation-computational resources. Science in digital econ-omy” (DICR-2017): Proceedings of XVI All-Russia Conference(4-7 of December, 2017) Novosibirsk. pp.142-149. Available at:http://elib.ict.nsc.ru/jspui/bitstream/ICT/1467/20 /paper16.pdf
РЕАЛИЗАЦИЯ АДАПТИВНОЙМОДЕЛИВВОДА И РЕДАКТИРОВАНИЯ ИНФОРМАЦИИНА ОСНОВЕ XSLT-ПРЕОБРАЗОВАНИЙ ДЛЯ
РАЗНОРОДНЫХ ДАННЫХ
Мухитова А., Жижимов О. Л.
Адаптивность, способность интерфейсов подстра-иваться под структуру и функциональность инфор-мационных источников, является одним из основ-ных признаков интеллектуальности информационнойсистемы. Разработка адаптивных графических веб-интерфейсов на основе XML-технологий позволя-ет визуализировать любую структуру файла XML-формата для возможности дальнейших манипуляцийпо вводу и редактированию данных. В работе рассмот-рена технология построения адаптивного графическо-го административного WEB-интерфейса для ввода иредактирования данных в разнородной информацион-ной среде на основе использования определений схемданных XSD с применением XSLT преобразований.Приводится пример реализации адаптивной моделиввода и редактирования информации в виде создан-ного прототипа редактора XML-записей. Данный ре-дактор, в архитектуре клиент-сервер, на стороне сер-вера генерирует пустую форму редактирования путемпреобразования модифицированной структуры XSDметодом XSLT и затем предоставляет на клиентскойчасти готовую HTML-форму с полностью необходи-мым инструментарием (java скрипты) для корректноговвода и/или редактирования неоднородных данных.
Received 10.01.19
178
A Model-Driven Development Approach forCase Bases Engineering
Nikita O. Dorodnykh and Alexander Yu. YurinMatrosov Institute for System Dynamics and Control Theory,
Siberian Branch of the Russian Academy of Sciences (ISDCT SB RAS)Irkutsk, Russia
[email protected], [email protected]
Abstract—The paper discusses application of a model-driven development approach for engineering knowledgebases of case-based reasoning decision support systems. Theconceptual models presented in XML-like formats are usedas the initial data. The problem statement, main stagesof the proposed approach and a tool are presented. Anillustrative example describes an educational task.
Keywords—model-driven development, case-based rea-soning, intelligent system, knowledge base, conceptualmodel
I. INTRODUCTION
The knowledge bases engineering for intelligent sys-tems remains a time-consuming. This process requiresthe involvement highly qualified specialists in domainareas (e.g., domain experts, analysts, programmers) andalso the use of specialized software. In this case, thedevelopment and use of software focused on non-programming users and implementing the principles ofvisual programming and automatic code generation arerelevant. There are examples of such software: ExSys,Visual Expert System Designer, and others. In mostcases, these systems are focused on a certain formalismof knowledge representation, in particular, logical rules.Software for conceptual modeling is another class of sys-tems that uses visual programming for knowledge basesengineering. Such systems provide the ability to createmodels in the form of concept maps, mind maps, fuzzymaps, entity-relationship diagrams, tree-like semanticstructures and schemes (fault trees, event trees), IDEFor UML models. Most of these systems are universalmodeling software and don’t take into account features ofintelligent system engineering, i.e. formalisms, languagesand program platforms, which are used in this area. Inthis connection, they provide a visual representation ofdomain-specific knowledge structures, but don’t supportan adequate knowledge codification (formalization) forknowledge representation languages.
The use of methods and approaches that imple-ment the principles of a Model-Driven Engineering ap-proach (MDE) (also known a Model-Driven Develop-ment, MDD) [1], [2] is a compromise in this case. Suchcompromise solutions [3] provide import (analysis andtransformation) of conceptual models, which describing
domain knowledge into knowledge base structures, andtheir specification in specialized editors for further execu-tion. Model transformations [4] are implemented duringthe import, for example, by using Transformation ModelRepresentation Language (TMRL) [5].
This paper discusses an example of a such compromisesolution, in particular, we propose an approach and itsimplementation on the basis of a Personal KnowledgeBase Designer (PKBD) platform [6]. Our proposalsprovide prototyping knowledge bases for case-baseddecision support systems. At that, concept maps andformalization from [7] are used as initial data.
II. STATE-OF-ART
Let’s consider main concepts and definitions in thefield of case-based reasoning and model-driven develop-ment (MDE/MDD) as a background for the research.
A. Case-Based Reasoning
Case-Based Reasoning (CBR) [8], [9] is a method-ology for decision making that reuses and adapts (ifnecessary) of the previously obtained solutions of similarproblems. This approach is based on the “by analogy”principle of decision making. The main concept is a case.The case is a structured representation of accumulatedexperience in the form of data and knowledge, providingits subsequent automated processing using specializedsoftware.
The main case features are the following:• A case represents special context-related knowledge
that allows to use these knowledge at the applicationlevel.
• Cases can be represented by various forms: cov-ering different time periods; linking solutions withproblem descriptions; results with situations, etc.
• Case captures only the experience that can teach (tobe useful), fixed cases can potentially help domainexpert (decision-maker) to achieve the goal, facil-itate its formulation in the future or warn him/hirabout possible failures or unforeseen problems.
A case structures units of experience. At the sametime, the used structure is a problem-specific. In general,a case structure includes two main parts:
179
• An identifying (characterizing) part describes theexperience by a way that provides to estimate thepossibility of its reuse in a particular situation.
• A learning part describes a lesson (learning knowl-edge, a solution) as a part of an experience unit,for example, a solution of a problem or its part, adecision proof (conclusion), an alternative or faileddecisions.
CBR applied to solving problems in various subjectdomains, including planning [10], diagnostics [11] andothers. There are some examples of CBR tools: CBR-Express, CasePoint, CasePower, Esteem, Expert Advi-sor, ReMind, CBR/text, ReCall, RATE-CBR, S3-Case,INRECA, and CASUEL.
B. Model-Driven Engineering
Model Driven Engineering (MDE) or Model-DrivenDevelopment (MDD) is a software design approachthat uses the information models as the major artifacts,which, in turn, can be used for obtaining other modelsand generating programming codes [2]. This approachenables programmers and non-programmers (dependingon the implementation) to create software on the basisof conceptual models.
The main MDD concepts are the following:• A model is an abstract description of a system (a
process) by a formal language. As a rule, models arevisualized with the aid of certain graphic notationsand serialized (represented) in XML.
• A metamodel is a model of a formal language usedto create models (a model of models).
• A four-layer metamodeling architecture is the con-cept that defines the different layers of abstraction(M0-M3), where the objects of reality are repre-sented at a lowest level (M0), then a level of models(M1), a level of metamodels (M2) and a level of ameta-metamodel (M3).
• A model transformation is the automatic generationof a target model from a source model with theaccordance of a set of transformation rules. Inthis case, each transformation rule describes thecorrespondence between the elements of a sourceand a target metamodels.
There are examples of a successful use of MDEfor development of database applications (e.g., ECO,for Enterprise Core Objects), agent-oriented monitoringapplications [12], decision support systems [13], [14],embedded systems (software components) for the Inter-net [15], rule-based expert systems [16].
C. Background
Proposals of the current research are based on theprevious works. In particular, development of case-basedexpert systems and knowledge bases for petrochemistryare presented in [11]. Model transformations, as well
as a specialized domain-specific declarative languagefor describing transformations of conceptual models arediscussed in [5]. Application of MDD principles for en-gineering knowledge bases of rule-based expert systemsare described in [3], the formalization of MDD for CBRis used from [7].
III. PROTOTYPING CASE BASES USINGTRANSFORMATION OF CONCEPTUAL MODELS
A. Formalization
Most of decision-making tasks can be described by aset of characteristics, and can be formalized as follows:
MTask = p1, ..., pn , pi ∈ Prop,Prop =
⋃Mi=1 pi, i = 1, N,
(1)
where MTask is a task model; pi is task properties(significant characteristics); Prop is a set of properties.
The task model from a CBR point of view is definedas follows:
MTaskCBR : ProblemCBR → DecisionCBR, (2)
where MTaskCBR is a task model in terms ofCBR; ProblemCBR is a task (problem) description;DecisionCBR is a problem decision, while:
ProblemCBR = 〈c∗, C〉 , C = c1, ..., ck , c∗ /∈ C, (3)
where c∗ is a new case; C is a case base.
DecisionCBR = d1, ..., dr ,di = (ci, si), ci ∈ C, si ∈ [0, 1],
(4)
where DecisionCBR is a problem decision in the formof a set of retrieved cases with similarities si.
Let’s formalize a case description:
ci =PropProblem
i , P ropDecisioni
, (5)
where PropProblemi is a identifying (characterizing) part
of a case; PropDecisioni is a learning part of a case. In
addition, each of these parts contains task properties:
PropProblemi = p1, ..., pm ,
P ropDecisioni = pm+1, ..., pn ,
P ropProblemi ∪ PropDecision
i = Prop,PropProblem
i ∩ PropDecisioni = ∅
(6)
The composition of the parts has a problem-specificcharacter.
There are many methods that can be used for caseretrieval [17] nearest neighbour, decision trees, etc. Themost popular method is the nearest neighbour, based onan assessment of similarity with the aid of different met-rics, for example, Euclidean, City-Block-Metric, etc. Theproposed approach uses the nearest neighbour methodand Zhuravlev metric [18] with normalisation:
180
si(c∗, ci) =
N∑
j=1
wjhi(p∗j , pij)/N,
hi(p∗j , pij) =
for quantitative
1, if |p∗j − pij | < ξ
0, else
for qualitative
1, p∗j = pij0, p∗j 6= pij
(7)where wi is an information weight, and ξ is a constrainton the difference between the property values. At thesame time, the normalisation (standardisation) is follows:
pik =(pik −min
kxik)/(max
kpik −min
kpik). (8)
B. Methodology
Generally, the methodology for expert systems en-gineering consists of the following main stages [19]:analysis and identification, conceptualization (structur-ing), formalization (representation), implementation andtesting. In turn, in accordance with the formalization [7]the process of intelligent systems engineering based onmodel transformations can be considered as the sequen-tial formation and transformation of models with varyingdegrees of abstraction:
• A computation-independent model (CIM) describeskey abstractions. Some problem-oriented notationsare used for its representation: UML, concept andmind maps, event trees. In turn, these models willbe serialized in XML-like formats: XMI (StarUML,IBM Rational Rose), CXL (IHMC CmapTools),ETXL (ET-Editor), etc.
• A platform-independent model (PIM) based on aCIM. This model depends on the formalism ofknowledge representation, but does not take intoaccount features of languages and the tools forimplementing these formalisms.
• A platform-specific model (PSM) takes into accountfeatures of languages and means for implementingformalisms.
At the same time, source codes or specifications ofknowledge bases and intelligent systems generated onthe basis of these models.
Figure 1 shows a comparison of the main stages ofcase bases and expert systems engineering and also theirrelationships with MDE artifacts.
C. Implementation
Implementation of the proposed methodology is madeon the basis of a Personal Knowledge Base Designer(PKBD) platform [6]. PKBD is designed for the end-users and provides the creation of knowledge basesand domain-specific editors by means of an implemen-tation of the main principles of the Model Driven-Architecture (MDA) [20] (a separate direction within the
Figure 1. The main stages of case bases and expert systems engineeringand their relationships with MDE artifacts.
MDE/MDD). The MDA assumes a clear definition ofthree levels (viewpoints) of the software representation:
• A computation-independent level that describes thebasic concepts and relationships of the subject do-main expressed in the form of conceptual models.Models of this level can be form automatically onthe basis of the analysis of XML-like formats, inparticular: XMI (XML Metadata Interchange) forUML class diagrams, CXL (Concept Mapping Ex-tensible Language) for concept maps, ETXL (EventTree Mapping Extensible Language) for event trees.
• A platform-independent level that represents a do-main model in the context of knowledge represen-tation formalism used, in particular, logical rules orcases.
• A platform-dependent level that represents a for-malized description of knowledge bases taking intoaccount features of a certain software platform (forexample, PKBD).
The PKBD architecture includes the main modules for:knowledge base management, knowledge representationlanguages support, dictionaries management, integrationwith CASE-tools, interpretation of models and the graph-ical user interface (GUI) generation. The interface isgenerated both in the form of pop-up windows andwizards. Wizards represent sequences of GUI forms,which segment and order the processes of entering andediting elements of a knowledge base. In particular, theuser is consistently asked to specify: the name of a casetemplate (used for display in the editor), the short name(used in the process of inference), the description andthe properties (slots) of the template when entering it;the property name, its short name, description, a type(string, symbol or number), a possible default value anda constrain for this value (more, less, equal, unequal)when entering a slot. The similar wizards are used whenentering and editing cases.
Let’s consider one of the educational examples of case-base engineering with the aid of PKBD from a course of“Information Systems Toolkits” of the Irkutsk NationalResearch Technical University (IrNITU).
181
IV. CASE STUDY
The educational task of case base engineering of astatic expert system for crystal minerals identificationsis considered as an example.
A crystal mineral model in the form of a conceptmap is constructed at the conceptualization stage withthe aid of IHMC CmapTools. A concept map fragmentcorresponding to a CIM is presented in Figure 2.
Figure 2. A fragment of a model of a “Mineral” concept.
Then, a PIM is formed in correspondence with the“Case” concept. In particular, all properties of the con-sidered concept added to the identifying (characterizing)part, while the learning part consists only of a mineralname.
Cases can be added to case base either manually usinga special wizard or by importing from text files. Testingthe developed case base carried out in a special wizardby building queries (see Fig. 3).
Figure 3. Preview query results and comparison of cases.
V. CONCLUSION
Knowledge base engineering, in particular, case bases engi-neering for CBR decision support systems remains to be a time-consuming process, which requires the use of specialized tools.The effectiveness of the use of such tools can be improvedby applying approaches based on generative programming andvisualization.
The paper proposes the MDE application for developmentof case bases. The conceptual models in the form of conceptmaps presented in XML-like formats are used as the sourcedata. PKBD (new module for case base engineering) is usedas a mean for implementing the proposed approach. The tooltested on educational tasks and on the problem of structuralmaterial selection for a petrochemical systems design [21].
REFERENCES
[1] A. R. Da Silva, Model-driven engineering: A survey supported by theunified conceptual model. Computer Languages, Systems & Structures,2015, vol. 43, pp. 139-155.
[2] M. Brambilla, J. Cabot and M. Wimmer, Model Driven Software Engi-neering in Practice. Morgan & Claypool Publishers, 2012.
[3] A. Yu. Yurin, N. O. Dorodnykh, O. A. Nikolaychuk and M. A. Grishenko,Designing rule-based expert systems with the aid of the model-drivendevelopment approach. Expert Systems, 2018, vol. 35, no. 5, pp. 1-23.
[4] K. Czarnecki and S. Helsen, Feature-based survey of model transformationapproaches. IBM Systems Journal, 2006, vol. 45, no. 3, pp. 621-645.
[5] N. O. Dorodnykh, S. A. Korshunov, N. Yu. Pavlov and A. Yu. Yurin, ModelTransformations for Intelligent Systems Engineering. Open Semantic Tech-nologies for Intelligent Systems, 2018, vol. 2, no. 8, pp. 77-81.
[6] M. A. Grishenko, N. O. Dorodnykh and A. Yu. Yurin, Software for ruleknowledge bases design: Personal Knowledge Base Designer. Processingof the 6th International Scientific and Technical Conference: Open Seman-tic Technologies for Intelligent Systems (OSTIS-2016), 2016, pp. 209-212.
[7] N. O. Dorodnykh and A. Yu. Yurin, About the specialization of model-driven approach for creation of case-based intelligence decision supportsystems. Open Semantic Technologies for Intelligent Systems, 2017, vol.7, pp. 151-154.
[8] A. Aamodt and E. Plaza, Case-based reasoning: foundational issues,methodological variations, and system approaches. Artificial IntelligenceCommunications, 1994, vol. 7, no. 1, pp. 39-59.
[9] R. Bergmann, Experience Management: Foundations, DevelopmentMethodology, and Internet-Based Applications. Springer. 2002.
[10] A. Popa and W. Wood, Application of case-based reasoning for wellfracturing planning and execution. Journal of Natural Gas Science andEngineering, 2011, vol. 3, no. 6, pp. 687-696.
[11] O. A. Nikolaychuk and A. Yu. Yurin, Automating the identification of me-chanical systems’ technical state using case-based reasoning. Processingof the 3rd International IEEE Conference Intelligent Systems, 2006, pp.687-696.
[12] J. M. Gascuena, E. Navarro and A. Fernandez-Caballero, Model-drivenengineering techniques for the development of multi-agent systems. Engi-neering Applications of Artificial Intelligence, 2012, vol. 25, no. 1, pp.159-173.
[13] J. Baumeister and A. Striffler, Knowledge-driven systems for episodicdecision support. Knowledge-Based System, 2015, vol. 88, pp. 45-56.
[14] R. Neto, P. J. Adeodato and A. C. Salgado, A framework for datatransformation in credit behavioral scoring applications based on modeldriven development. Expert Systems with Applications, 2017, vol. 72, pp.293-305.
[15] J. Canadas, J. Palma and S. Tunez, InSCo-Gen: A MDD Tool for WebRule-Based Applications. Web Engineering, 2009, vol. 5648, pp. 523-526.
[16] M. A. Nofal and K. M. Fouad, Developing web-based Semantic and fuzzyexpert systems using proposed tool. International Journal of ComputerApplications, 2015, vol. 112, no. 7, pp. 38-45.
[17] L. R. Da Mantaras, D. Mcsherry, D. Bridge, D. Leake, B. Smyth, S. Craw,B. Faltings, M. L. Maher, M. T. Cox, K. Forbus, M. Keane, A. Aamodt andI. Watson, Retrieval, reuse, revision and retention in case-based reasoning.Knowledge Engineering Review, 2005, vol. 20, no. 3, pp. 215-240.
[18] I. Yu. Zhuravlev and I. B. Gurevitch, Pattern recognition and imagerecognition. In Pattern recognition, classification, forecasting: Mathemat-ical techniques and their application. Nauka: Moscow, 1989, vol. 1, pp.5-72. (In Russ.)
[19] P. Jackson, Introduction to expert systems, 3rd ed. Harlow: Addison-Wesley, 1999.
[20] MDA Specifications. Available at: http://www.omg.org/mda/specs.htm (ac-cessed 2018, Dec).
[21] A. F. Berman, G. S. Maltugueva and A. Yu. Yurin, Application of case-based reasoning and multi-criteria decision-making methods for materialselection in petrochemistry. Proceedings of the Institution of MechanicalEngineers, Part L: Journal of Materials: Design and Applications, 2018,vol. 232, no. 3, pp. 204-212.
РАЗРАБОТКА ПРЕЦЕДЕНТНЫХ БАЗ ЗНАНИЙ СИСПОЛЬЗОВАНИЕМMDE-ПОДХОДА
Дородных Н.О., Юрин А.Ю.
В работе рассмотрено применение модельно-управляемого подхода для создания прецедентных баззнаний. Концептуальные модели, представленные в XML-подобных форматах, использованы в качестве исходныхданных. Представлена формализованная постановка задачи,основные этапы подхода и инструментальное средство.Приведен пример решения учебной задачи.
Received 17.12.18182
Knowledge Management System and DigitalTransformation of CompanySergey Gorshkov
Roman ShebalovTriniData
Abstract—The article considers the different approachestowards modelling and implementation of knowledge man-agement systems in the context of the enterprise’s digitaltransformation. The methodology is based on the conceptu-alization of the domain using Semantic Web technologies,i.e. OWL and SPARQL standards, and semantic reasoners.The purpose of this paper is to present the results ofknowledge management (KM) system design. Semantic-based architecture of search engine is proposed as aninstrument for strict searching and corporate knowledgeexploration. The KM-system deployment is supposed to beas a checkpoint of digital transformation of a companywhich means that the company is eager to earn money fromdata considered as an asset. As a whole the advantages ofontology-based KM-system are: minimizing the mistakesmade by the employees; fast adaptation and involving theemployees recently hired by the company; improving thespeed and quality of analytics in company; preserving theknowledge which could be lost because of staff retirement.
Keywords—knowledge, ontology, knowledge manage-ment, knowledge management system, information mod-elling, information management, Semantic Web
I. INTRODUCTION
The knowledge system is an essential requirementfor business success of the company. Some companiesemploy data stewards for managing the data as thecompany’s values. We consider knowledge as the infor-mation directly used in company’s everyday processes.In contrast, the data is the stored information whichhas to be extracted and interpreted for use. That meansin common that the data stewards are to consolidateand operate all the accessible data resources, to pro-duce knowledge from data. This activity literally means‘knowledge management’.
The purpose of this paper is to present the results ofknowledge management (KM) system design. This KMsystem is aimed to aggregate, normalize and transformcorporate data in accordance with the ontology model ofdomain.
II. METHODOLOGY
A. Our KM Designing Approach
KM considers the deliberate structuring of knowledgeand presenting it for users by request. The functions ofKM-system are:
• accumulation of useful knowledge;• informing the company’s employees of the accessi-
bility of the knowledge;• presenting tools for structuring, transforming and
linking pieces of knowledge of different sources.The KM-system is supposed to become a digital
‘brain’ of a company delivering necessary support duringthe staff hiring and retirement, collecting the best prac-tices for preventing the mistakes and decisions makingsupport. However, the term ‘KM-management system’is often used for the IT-tools for specific tasks. Forexample, Service Desk, Help Desk or even social mediaor the huge unstructured collections of the files may beconsidered as enterprise KM-systems. These solutions donot manage the knowledge, they just collect the pieces ofinformation according to the strict structure. Consideringthe limits and weakness of traditional ‘KM-tools’ it ishighly important to establish a new conception of KM-system. We suggest that the solutions bearing this nameshould be based on the following principles:
• including the conceptual (ontology-driven) domainmodel which is based on the terms and vocabularyused in the company;
• including software tools for model management;• addressing as many sources as possible (internal and
external ones) and index the content;• letting the users ask questions in terms of conceptual
model and answer them addressing to the datasources if needed;
• including some tools for spreading the best enter-prise’s practices among the employees as well asinforming the latter about the knowledge elementsthey may need;
• containing the tools to input the information whichcannot be placed into any database of a company.
Usually such information corresponds to the employ-ees’ experience and therefore must be saved because ofits high business value. The KM-system shall act as an‘expert’ which answers the users’ questions better thanany human. It is a ‘lens’ through which a user may lookat any piece of corporate data and effortlessly transformit into the task-relevant knowledge. Implementation ofthe KM-system in the above presented sense may appear
183
to be impossible for CEOs in general. But the recentdecade achievements in IT and semantic technologiesin particular make this task quite reachable. As forthe prerequisites they are OBDA paradigm (OntologyBased Database Access) [1], corporate sources metadataannotation projects [2], [8], and conceptual formalizationof natural language projects [3], [4], [5], [6]. SemanticWeb technologies play the main role in the machinestorage and pro-cessing of the formalized conceptualmodels and data. OWL and SPARQL standards, semanticreasoners are the core components of Semantic Webstack. Their advantages are: linked data operation, datamanagement flexibility, automated reasoning and infer-ence for consistency control and data enrichment.
B. Current OWL-Practices for KMOne of the most OWL-involved industries in the world
is oil and gas. All the worldwide companies use theowl modelling in geology, field managing, reequippingthe wells and petroleum extracting analysis. The mostrelevant solutions are i-Field by Chevron, Smart Fieldby Shell, Integrated Operation for the High-North byinternational consortium of 22 members, Field of theFuture by BP, Integrated Production Management byExxonMobil, and Intelligent Field Program by SaudiAramco. In addition to efforts from major oil and gasorganizations, service organizations like Baker Hugheshas devised novel approaches for capturing, encoding,and provisioning of actionable knowledge from expertsdeployed in the field. As part of the data managementeffort, it is important to adopt effective record keepingand data curation strategies that have been extensivelystudied and addressed in other data-intensive disciplines[2], [7].
The ontology models consider reducing or even avoid-ing the ambiguity of the information referred to oil andgas exploration. It is possible because of semantic en-richment of the ontology model with the metaproperties.Especially in Petroleum Geology, properties like identityand unity can help in defining what exactly are theentities of reality that are being modeled in the databaseand also provide a good support to integrate models in theseveral scales of analysis (microscopic, well, reservoir,basin scales) into the petroleum chain [9].
The ontology approach is quite flexible. So it can bemanaged as ‘stable’ data as ‘real-time’ data which istransferring from multiple sources in variable-length timeintervals. There are some solutions in oil and gas com-panies (Chevron, for example) focused to the monitoringof petroleum infrastructure. The current status of the oilwells is presented in live mode. Being gathered suchinformation is recombined and classified by the logicalrules based on the ontology of events. As a result theinformation about the well status is delivered to a useras knowledge relevant to the specific moment of time[10].
As presented above the ontology modelling in com-bination with Semantic Web technologies can be imple-mented as a core of high-rated IT-solutions.
III. IMPLEMENTATION
We stated above that we see a corporate KM-system asan expert system which can find any information requiredby the user and to answer user’s queries. The user expe-rience starts with the query formulation interface. It maybe implemented as a controlled natural language inputtool, which hints the user the appropriate terms whichcan be used in request text and prevents from entering thesentences which sense may not be recognized. Anotheroption is a graphical request constructor, allowing user tocompose the query conditions using visual blocks, eachof which represents one of the interrelated informationalobjects participating in the search.
The result of the first step is a query which includeconditions on the several entities, for example: the com-panies which has ordered a survey which costs more thanUSD100,000. To answer this query, the system has toidentify the entities (company, survey), their properties(cost) and relations (has ordered), find the sources ofappropriate information in the corporate data storages,extract the objects matching the conditions and finallycomplete the query answer.
Obviously, it is impossible to develop a physicalrepository specially for KM-system. The Logical DataWarehouse is an appropriate architecture which allowsusers to get access to any information, independentlyof the physical server/database where it is stored. Log-ical Data Warehouse has to determine the informationrequired to fulfill the user’s search request, gather datafrom sources, aggregate it and present the results. Thesearch requests must be formulated using the master-dataset. It means that when a user refers to some businessobject (a customer, an asset etc.) in the search request,this object has to be identified according to the masterdata. It intends that MDM-system and KM-system shouldbe integrated or KM-system should become a source ofthe master data. As KM-system retrieves the data fromdifferent sources (like an aggregator or a feed-reader) itshould be able to access these sources, which can bemediated by ESB. ESB (Enterprise Service Bus) allowsto read the information of as many company’s databasesas there are and deliver it to KM-system in order toanswer the users’ questions. Due to such an integrationKM-system may become one of the key components ofthe IT-infrastructure of a company [Fig.1].
The semantic approach towards data conceptualization(including search and transformation) is very flexible.So, it may be used for narrow or wide purposes aswell. The opportunity to process strict and complexsearch requests makes ontological tools very useful inthe enterprise data management. They allow to extract
184
Figure 1. KM-system among the IT-infrastructure components
all the data answering the specific search request, butnot only “the most relevant” or “the first n-entities” asthe usual Internet search machines do. This strict searchcan be implemented with the ontology-structured datamodel. In other words, the search should be ontology-driven. Classes, objects and attributes of the ontologyare the entities which represent the domain of company’sbusiness activities. Therefore the ontology model turns tobe a framework of KM-system. The ontology data modellinks the entities and operates during the search requestprocedures. The basics of this framework can be picturedas follows: [Fig.2].
Figure 2. Searching by KM-system.
Presented scheme of data management procedures putsthe KM-system into the center of an enterprise IT-
landscape and provides a lot of benefits for the users,mostly because of the instruments of linking, aggregatingand searching knowledge. These instruments are beyondthe traditional (non-ontology) methods. The semanticsearch can easily deal with different types of data (texts,graphics, sheets and so on). Also the semantic enrich-ment can be managed by multiplying the connectionsbetween entities during the automatic text mining.
The KM-system can be involved in several scenarios:
• complex analytical system deployment which pro-cesses the data in a variety of dimensions, such aspurposes of business activities, staff, territories ofoperations and so on;
• managing the arrays of information according tothe selected task (for example, all the company’sdocuments about specific type of asset);
• searching for the analog of entities in corporate data(i.e. documents, objects, processes etc.);
• accessing the enterprise data by the managementand/or staff through a single access point;
• integration of the different enterprise software (doc-ument flow system, consumer relationship manage-ment system – CRM, personnel management sys-tem, project management system and any others);
• optimization of staff’s efforts for collecting, valida-tion and transforming the information;
• supporting the exchange of employees’ experienceand collecting pieces of one’s experience in a struc-tured and catalogized way.
The KM-systems addressed to these scenarios arebeing implemented in Russia [11] and other countries[12].
IV. DISCUSSION
According to our experience, there are some restric-tions which usually make some difficulties to bring theKM-system into company’s life. Firstly, it is due to lackof CEOs’ readiness to start the process. Secondly, it isbecause of the low level of IT-infrastructure developmentin a company. As for the first restriction, it is possibleto inform or even educate the managers and CEOs aboutthe advantages of the KM-approach. As for the secondone, the most effective way is employing an IT-companyas a contractor for KM-system development. And laterthe local IT-specialists will have an opportunity to en-rich their competence in ontology design. Another fieldfor discussion about KM-system addresses towards theeconomy sector limits. Can ontology-based KM-systembe effectively used in all of them? We suppose that itis directly appropriate for the companies of industrialsector (focused on energy, engineering, machinery andso on). For service companies or social-media companiesthe KM-system should probably be accompanied by themachine learning and predictive analytics technologies.
185
Finally, KM-system implementation requires the devel-opment of the ontology model which requires collab-oration of the analysts aware of conceptual modelingmethodologies and domain experts, which is a resource-intensive task for an organization.
V. CONCLUSION
The most of the currently implemented corporate KM-systems are unable to transform the vast amounts of cor-porate data into an actionable knowledge. To accomplishthis task, the view of a corporate KM-system shouldbe shifted toward semantic information processing. Themodern KM-system shall operate the meaning of in-formation, not just the text or database records. Theontologies and Semantic web technology stack are themost appropriate tools to accomplish this task.
We claim that the functional features of KM-systeminclude:
• supporting the structural management of data modelwhich is based on conceptual domain model. Thedata model should have an implemented multi-pointview approach [13].
• transforming all the accessible corporate informa-tion in correspondence with the structure of datamodel. This is a way to utilize it not as data (i.e.items stored in a warehouse) but as knowledge.The corporate end-users will be able to operatesuch knowledge in active and proactive mode intheir professional activities and without exerting anyinterpretation effort.
The KM-system deployment is supposed to be as acheckpoint of digital transformation of a company whichmeans that the company is eager to earn money fromdata considered as an asset. As a whole the advantagesof ontology-based KM-system are:
• minimizing the mistakes made by the employees;• fast adaptation and involving the employees recently
hired by the company;• improving the speed and quality of analytics in
company;• preserving the knowledge which could be lost be-
cause of staff retirement.The expenses needed for the KM-system deployment
should be considered as the strategic investments for thedigital future of a company.
REFERENCES
[1] E.Kharlamov et al. “Ontology Based Data Access in Statoil”, WebSemantics: Science, Services and Agents on the World Wide Web.vol. 44, pp. 3–36. 2017.
[2] C.Chelmis, J.Zhao, V.Sorathia, S.Agarwal, V.Prasanna “Towardsan Automatic Metadata Management Framework for Smart OilFields”, SPE Economics and Management. vol. 5, is. 01. 2013.
[3] N.V.Loukcshevitch, B.V.Dobrov “Proektirovanie lingvistich-eskikh ontologii dlya informatsionnykh sistem v shirokikh pred-metnykh oblastyakh” [Modelling of Linguistic Ontologies forIT-solutions in Various Spheres], Ontologicheskoe modelirovanie[Ontology of Designing], 2015. Vol.5. no 1., pp.47-69.
[4] V.F. Roubashkin Ontologicheskaya semantika. Znaniya. On-tologii. Ontologicheski orientirovannye metody informatsionnogoanaliza tekstov [Ontology Semantics. Knowledge. Ontologies.Ontological Methods of Text Analysis]. Moscow, Fizmatlit, 2013.348 p.
[5] V.F.Roubashkin Predstavlenie i analiz smysla v intellektual’nykhinformatsionnykh sistemakh [Representation and Analysis ofMeanings in Intellectual IT-systems]. Moscow, 1989, Nauka. 192p.
[6] Ph.Cimiano “Ontology Learning and Population”, Text. Algo-rithms, Evaluation and Applications. Springer. 2006.
[7] Digital Oilfield Ten Years On: A Literature Review. Energysys.2016. p.11.
[8] R.Werlang, M.Abel, M.Perrin, J.Carbonera, S.Fiorini Ontologicalfoundations for petroleum application modeling. Research Gate.2014. https://tinyurl.com/ybp73euy (accessed 2018, Oct).
[9] M.Abel, M.Perrin, J.Carbonera, L.Garcia Ontologies and datamodels: essential properties and data modeling for petroleumexploration. Research Gate. 2016. https://tinyurl.com/yd4l8yeu(accessed 2018, Oct).
[10] Zhu T., Bakshi A., Prasanna V., Cutler R., Fanty S. SemanticWeb Technologies for Event Modeling and Analysis: A WellSurveillance Use Case. SPE Intelligent Energy Conference andExhibition. Utrecht, The Netherlands. 2010. p.2-3.
[11] https://serge-gorshkov.livejournal.com/48835.html (accessed2018, Oct).
[12] http://www.kmworld.com/Articles/News/KM-In-Practice/Oil-company-integrates-service-and-information-management-128131.aspx (accessed 2018, Oct).
[13] S.Gorshkov, S.Kralin, M.Miroshnichenko Multi-viewpoint On-tologies for Decision-Making Support. Springer, 2016. Knowl-edge Engineering and Semantic Web. Vol. 649 of the seriesCommunications in Computer and Information Science. pp 3-17.
СИСТЕМА УПРАВЛЕНИЯ ЗНАНИЯМИ ИЦИФРОВАЯ ТРАНСФОРМАЦИЯ КОМПАНИИ
Сергей Горшков, Роман ШебаловАннотация: Статья посвящена подходам к проектиро-
ванию и внедрению систем управления знаниями в усло-виях развития цифровых трансформационных процессов вразличных отраслях экономики. Методология основана напредставлении концептуальных моделей предметных обла-стей с использованием технологий Semantic Web (стандартпредставления концептуальных моделей OWL, язык доступак графовым базам данных SPARQL, машины логическоговывода). Цель статьи- представление результатов проекти-рования системы управления знаниями.
Предлагается собственная архитектура поисковой си-стемы, основанной на семантических технологиях. Даннаясистема рассматривается в качестве инструмента по точномупоиску и извлечению корпоративных знаний. Реализация по-добной поисковой системы выступает в качестве "контроль-ной точки"в процессе цифровой трансформации компании,что предполагает извлечение прибыли в ходе обработки дан-ных, которые рассматриваются как актив компании. В целомпреимуществами систем управления знаниями, основанныхна онтологиях, являются: уменьшение количества ошибок состороны работников; быстрая адаптация новых работников иих активное включение в деятельность компании; повышениескорости и качества аналитики в компании; сохранениезнаний, которые могут быть утрачены в связи с уходомработников из компании.
Received 09.12.18
186
The Next Stage of Industry 4.0: From Cognitiveto Collaborative and Understanding Agents
Valery B. TarassovBauman Moscow State Technical University
Moscow, [email protected]
Abstract—The paper considers the role of artificial cogni-tive, collaborative and understanding agents in developingIndustry 4.0 initiative. Primarily, a proposal of using bothopen semantic and pragmatic intelligent technologies for In-dustry4.0 is justified. The evolution of Industry 3.0 and thefirst International Program on Intelligent ManufacturingSystems are analyzed as forerunners of Industry 4.0. Somebasic ideas and principles of Industry 4.0 are clarified, itsenabling technologies are presented. The thesis about enter-prise total agentification is formulated. A possible solutionof the problem how to construct artificial understandingagents is suggested. Finally, three basic ways of developingnew generation technologies for Industry 4.0 are discussed
Keywords—Artificial Intelligence; Intelligent Agent; In-dustry 4.0; Cyberphysical System; Internet of Things;Collaborative Robot; Enterprise Agentification
I. INTRODUCTION
Nowadays the worldwide initiative called Industry 4.0[1-5] becomes a main challenge for developing openadvanced semantic and pragmatic technologies in mod-ern Artificial Intelligence. An important justification forthe relevance of this thesis is the organization of theFirst International Conference on Industry 4.0 and AItechnologies; it will be held in August 2019 at Cam-bridge University, United Kingdom. Among its hot prob-lems such topics as AI-based hardware, virtual agents,clustering, machine learning, deep learning platforms,evolutionary computations, speech recognition, naturallanguage generation, knowledge representation and rea-soning, text analytics, intelligent simulation and robotics,data interpretation and analysis, including graph andnetwork approaches to data mining, are mentioned.
In Russia the First Workshop «Industry 4.0 Strategy,Internet of Things and Ambient Intelligence» took placeat the conference «Intelligent Systems and Computer-Integrated Manufacturing», which was organized jointlyby Bauman Moscow State Technical University andRussian Association for Artificial Intelligence in January25-26, 2019. Below we will tend to establish some linksbetween technologies of Industry 4.0 and OSTIS project.
The OSTIS project [6,7] has been initiated in orderto develop open semantic technologies of designingintelligent systems. We suggest its complementation byoutlining open pragmatic technologies for intelligent
agents, rising to the ideas of the «Father of Pragmatism»Ch.S.Peirce [8].
Let us point out that in information theory and semi-otics a clear difference between semantics and pragmat-ics is made. Semantics expresses the relation betweenmessage and its author or sender, whereas pragmaticsconsiders the value of message for its user in the contextof his goal achievement. By taking pragmatics rules, wecope with many-valued or even uncertainty-valued (theterm coined by V.V.Martynov [9,10]) reality of naturallanguage and select some current value – the basicone for a given time. To differ from semantics whichhas no addressee, pragmatics takes into account such aspecial addressee – interpreter. These considerations arequite relevant while building intelligent technologies forIndustry 4.0.
The aim of the paper consists in reviewing the-state-of-the-art in modern technologies for Industry 4.0, as wellas analyzing new intelligent, cognitive, social approachesto implement NBICS convergence concept [11] for thenext stage of Industry 4.0.
Primarily, the difference between Industry 3.0 andIndustry 4.0 is discussed, and the shift from pure manu-facturing to a family of info-communication industrialtechnologies for digital, virtual, smart enterprises isshown. Then nine basic components of Industry 4.0are considered, including Cyberphysical Systems andInternet of Things, Big Data Analytics and Cloud Tech-nologies, Intelligent Simulation, Augmented Reality andCollaborative Robots. Finally, the problem of total en-terprise agentification based on both physical and virtualartificial agents able to «understand» required behaviorpatterns in a specific industrial situation is faced.
II. WHAT IS INDUSTRY 3.0?
A. Evolution of Industrial Revolutions: Four Big Jumps
The sequence of industrial revolutions is shown in Fig-ure 1 (see [12]). The First Industrial Revolution (Industry1.0) was deployed in XVIII-XIX centuries (more exactly,between 1760 and 1820) and could be viewed as thedawn of industrialization. There was the transition fromhandcraft to machine-based human work, from agrarianand rural to mainly industrial and urban society. The iron
187
and textile industries were its locomotives; basic symbolswere water power, steam engines, mechanization throughspinning mills.
The Second Industrial Revolution, which is also calledthe Technological Revolution (Industry 2.0), took placebetween 1870 and 1914, just before World War I. Itwas a period of rapid industrialization, including boththe growth of pre-existing industries and expansion ofnew ones, such as steel, oil and electricity. The electricpower was used to create the mass production.
Advances in manufacturing and production technol-ogy enabled the widespread adoption of technologicalsystems such as telegraph and railroad networks. TheXXth century symbols of Industry 2.0 were early fac-tory electrification and the assembly lines (first of all,automobile production lines of Henry Ford).
The Third Industrial Revolution in the last third of XXcentury was the introduction of electronics, computersand automation in manufacturing. So a big industrialrobot for assembly was selected as its typical face.
Figure 1. Symbolic Representation of Four Consequent IndustrialRevolutions
While Industry 3.0 faces the problem of automatingsingle machines and technological processes with usingcomputers and electronic devices, Industry 4.0 focuseson the end-to-end digitization of all physical assets andtheir integration into digital ecosystems with value chainpartners. Digitization means the process of convertinginformation in a digital form; the result is called digitalrepresentation. Here the keyword is «Ubiquitous Digiti-zation», i.e. digitization and integration of both verticaland horizontal value chains, digitization of product andservice offerings, digitization of business models andcustomer access, and so on.
In a wide sense, Industry 4.0 encompasses both anew industrial enterprise vision and its keynote missionin the age of digital economy. The basic principle ofIndustry 4.0 states: by connecting machines, work piecesand systems, businesses are creating intelligent networksalong the entire value chain that can control each otherautonomously.
To show necessary prerequisites for Industry 4.0, letus consider the evolution of Industry 3.0 technologies.
B. On Basic Steps of Industry 3.0.
Manufacturing systems in the era of mass productionwere based on homogeneous automated lines operatingin a stable, well-defined environment. Flexible technolog-ical modules are heterogeneous and more efficient; theyinclude machines, instruments, manipulators and roboticsystems.
The arrival of Flexible Manufacturing Systems (FMS)means a further increase of complexity. All the compo-nents of flexible modules are present; besides, automatedstorage and retrieval systems, transportation systems,planning and control systems, local computer network,and other tools are included. Here FMS are good ex-amples of complex, heterogeneous, highly integratedsystems with different subsystems. In particular, flexibletechnological subsystems, flexible transport subsystems,flexible measurement-information subsystems are worthmentioning. Various robots equipped with their owncomputer systems can also be viewed as technologicalmodules. For instance, transportation robots, welding orassembly robots, stackers-robots are widely used in FMS.A typical example of educational-training Denford FMSis given in Figure 2.
Figure 2. Outward Appearance and Architecture of Denford FMS
A flexibility of complex system means its capacity ofrapidly react to environment changes and quickly adaptto these changes. In case of manufacturing system theflexibility supposes the capacity to quickly adjust withoutconsiderable expenses both to make new or modernizedproducts and introduce new technological processes withnew equipment.
The concept of Computer-Integrated Manufacturing(CIM) concerns such complex industrial systems as job-shop, enterprise, network of enterprises, where all oper-ations with information flows for all the phases of man-
188
ufacturing are based on computer technologies [13]. Itis worth noticing that CIM is more sophisticated systemwith respect to FMS. Apart FMS subsystems, it includesCAD/CAM/CAE system, PLM (Product Lifecycle Man-agement), MRP II (Second Generation ManufacturingResource Planning) standard, and so on.
Conventionally, CIM was viewed as a considerablepart of product lifecycle from the expression of theneed in this product to its launch to the market; thestage of product use was considered as an external onewith respect to CIM. Later on, both complex productmaintenance and its demolition became the trouble of itsproducer. In 1990’s a new concept of CIM appears thatencompasses all the product lifecycle, where the idea oflifecycle inversion after product demolition and invertedmanufacturing realization is crucial [14].
Now it is clear that CIM supposes Enterprise In-tegration [15]. Moreover, the idea of MetaCIM asComputer-Integrated Manufacturing in Networked, Vir-tual, Computer-Integrated Enterprises has been suggested[16]. Thus, a multi-dimensional computer-based inte-gration around the triple «product lifecycle–enterpriselifecycle–industrial knowledge lifecycle» has been con-sidered.
The next natural step consists in organizing distributedmanufacturing systems in virtual enterprises [17,18].
The evolution of Industry 3.0 production systems isdepicted in Figure 3 (see [19]).
Figure 3. Evolution of Computer-Integrated Manufacturing Systemsin XX Century
C. Intelligent Manufacturing Systems: First Results
The first step in making manufacturing intelligent wasthe International Program «Intelligent ManufacturingSystems» (IMS) which was started in the mid 1990’s (see[20,21]). The objective of this program was the creationof new generation manufacturing systems and technolo-gies by performing global intercontinental joint projects.These big projects, such as GLOBEMAN’21 (the ab-breviation of Global Manufacturing in XXI century)[21], Next Generation Manufacturing Systems, GNOSIS,
Holonic Manufacturing Systems faced many aspects ofautomated, integrated, intelligent manufacturing.
Primarily, the topics of IMS program were divided intofive groups: 1) modeling and management of total prod-uct lifecycle; 2) analysis and development of productionand business processes for enterprises of different in-dustries; 3) enterprise strategy planning and engineeringtechniques and tools; 4) human, organizational, socialfactors of production; (5) virtual and extended enter-prises.
Let us focus on GLOBEMAN’21 project. It wasdevoted to the problems of enterprise integration andproduct lifecycle modeling for global manufacturing inXXI century. The following results are worth analyzing[21]: a) design of direct and inverse product lifecyclemanagement systems; b) creation of new technologies ofintelligent simulation, decision support and productionmanagement for the enterprise networks; c) developmentof innovative CIM architectures in the enterprise net-works to obtain world class products; d) remote customerservice and support by using the information aboutreal customers needs and real products manufacturedby plants located in different parts of the world; e)more complete and deep understanding of new trendsin manufacturing related to advanced information andcommunication technologies.
It is worth stressing that within the IMS program somebasic knowledge engineering problems were faced andsolved: (1) acquisition of manufacturing knowledge andexperience; development of large and distributed knowl-edge bases; (2) data mining and knowledge discovery inmanufacturing; (3) implementation of heavyweight onto-logical models for virtual enterprises (see ToVE project[22]); (4) creation of intelligent information technologiesfor production management; (5) design of innovativeenterprises and manufacturing systems on the basis ofArtificial Life approaches and bionic (swarm cognition)algorithms.
So GLOBEMAN’21 project was performed by aninternational consortium to develop and demonstrate theenterprise integration tools and methods. Its purpose wasenabling manufacturing enterprise by new technologiesto form a mission oriented project organization, i.e. avirtual corporation, for networked manufacturing busi-ness. Both industrial and university partners from variouscountries and even continents took part at the project.These partners formed virtual organization. It was animportant step on the way to Industry 4.0.
Virtual organizations can be divided into virtual cor-porations and virtual partnerships. A virtual corporationis loosely coupled enterprise which is formed by manypartners to fulfill a difficult mission requiring sharedresources or organize world class production. Variousexamples of virtual partnerships can be bound in socialnetworks, such as Twitter, Instagram, Linkedin.
189
III. INDUSTRY 4.0 AND ITS COMPONENTS
The term Industry 4.0) firstly appeared at HannoverMesse in 2011 as an outline of German industrial per-spectives [1]. Nowadays the concept of Industry 4.0 hasspread far beyond Germany and is widely used all overthe world. The similar initiatives are called: Industrial In-ternet in the USA, High Value Manufacturing Catapult inthe United Kingdom, Usine du Futur in France, Fabbricadel Futuro in Italy, Smart Factory in Netherlands, MadeDifferent in Belgium, Industrial Value Chain Initiativein Japan, Made in China 2025, National TechnologyInitiative in Russia, and so on.
In 2015, McKinsey [23] defined Industry 4.0 as «thenext phase in the digitization of the manufacturing sector,driven by four basic factors: a) an astonishing rise in datavolumes, computational power and connectivity, in par-ticular, low-power wide-area networks; b) the emergenceof analytics and business-intelligence capabilities; c) newforms of human-machine interaction such as touch inter-faces and augmented-reality systems; d) improvementsin transferring digital instructions to the physical world,such as advanced robotics and 3D printing».
The most important idea of Industry 4.0 is the fusionof the physical and virtual worlds [1] provided by Cy-berphysical Systems (CPhS) [1,24]. The emergence ofCPhS means the inclusion of computational resourcesinto physical-technical processes. In others words, em-bedded computers and networks monitor and control thephysical-technical processes with feedback loops, wherephysical processes affect computations and vice versa.Within modular smart factories, physical processes aremonitored, virtual copy of physical world is created, andwell-timed decentralized decisions are made.
Let us note that CPhS are mechatronic systems en-hanced by advanced tools of data/knowledge acquisition,control and communication. Their components continu-ously interact, providing CPhS self-adjustment and adap-tation to changes. It is obvious that CPhS are crucialfor production digitization. Here work pieces, devices,equipment, production plant and logistics componentswith embedded software are all talking to each other.Smart products know how they are made and whatthey will be used for. Thus, both production machinesand equipment and products become cognitive agentsinvolved into manufacturing and logistics processes.
A vision of smart factory as a system of CPhS is givenin Figure 4.
There are five basic principles for implementing Indus-try 4.0: 1) Interconnection; 2) Information Transparency;3) Total Interoperability; 4) Decentralized Decisions; 5)Technical Assistance. Here Interconnection is viewedas the ability of both people and machines, devices,sensors communicate with each other via the Internetof Services and Internet of Things. Interconnectivitysupposes Information Transparency: it allows to collect
Figure 4. Basic Components of Smart Factory
immense amounts of data and information from all pointsin manufacturing processes and make more adequatedecisions. Decentralized Decisions are tightly connectedwith CPhS, which are able to perform their tasks asautonomously as possible. Only in case of emergencydecision-making is delegated to a higher level. Tech-nical Assistance primarily concerns the simulation ofmanufacturing process in virtual world; more generally,it is the ability of artificial agents to support humanagents, in particular, by performing unsafe, unpleasant ortoo exhausting tasks. Total Interoperability is understoodas the ability of industrial system to work with otherproducts and systems without any restrictions.
Nowadays, BCG’s nine basic technologies for Industry4.0 are usually considered [25] (see Figure 5). Let usbriefly analyze these technologies.
Figure 5. Nine Key Technologies for Industry 4.0 (by BCG)
190
A. Enterprise Integration and Engineering
The implementation of Industry 4.0 is closely relatedto enterprise engineering problems [26], in particular,strategic and ontological engineering [27]. With Industry4.0, companies, departments, functions, and capabilitieswill become much more cohesive, as cross-company, uni-versal data-integration networks evolve and enable trulyautomated value chains. Both horizontal and vertical en-terprise integration takes place [3]. On the one hand, theinitiative Industry 4.0 means digital representation andvertical integration of basic processes across the entireenterprise, from product development and purchasing,through manufacturing, logistics and service. On theother hand, horizontal integration goes beyond the inter-nal enterprise operations by involving both suppliers andcustomers together with all other value chain partners.It includes technologies from track and trace devices toreal time integrated planning with execution.
B. The Internet of Things
The Internet of Things (IoT) is a vision where everyobject in the world has the potential to connect to theInternet and provide their data so as to derive actionableinsights on its own or through other connected objects.The Internet of Things allows people and things to beconnected anytime, anyplace, with anything and anyone,ideally using any path/network and any service [28]. Theappropriate technologies open new, wide opportunitiesfor engineering networked enterprises.
The term «Internet of Things» was first coined byKevin Ashton, the founder and head of Auto-ID Centerin MIT, in 1999. As he stated, «the IoT has the potentialto change the world, just as the Internet did – maybeeven more so» [29]. It will comprise many billionsof Internet-connected objects (ICOs) or «things» thatcan sense, communicate, compute, evaluate, interpretand potentially actuate, as well as have intelligence,multimodal interfaces and social ability.
Gartner defines IoT as «the network of physical objectsthat contain embedded technology to communicate andsense or interact with their internal states or the externalenvironment» [30]. In [31] it is specified as a dynamicnetwork of uniquely identified objects that communicatewithout human interaction by using IP. This infrastruc-ture, possessing self-configurating capabilities, is basedon standard and interoperable communication protocols,where physical and virtual things have identifiers andphysical attributes, use intelligent interfaces and aretightly interconnected.
Nowadays, such communication and network tech-nologies as IPv6, web-services, Radio Frequency IDen-tification (RFID) and high speed mobile 6G Internetnetworks are employed.
The IoT incorporates basic concepts from pervasive,ubiquitous, cognitive computing, which have been evolv-
ing since the late 1990’s and have now reached somelevel of maturity.
Over the Internet of Things, cyber-physical systemscommunicate and cooperate with each other and withhumans in real-time both internally and across organi-zational services offered and used by participants of thevalue chain.
The IoT is an enabler to many application domains in-cluding intelligent manufacturing, product lifecycle man-agement, smart logistics and transportation, aerospaceand automotive industries. Society-oriented applicationsof IoT include smart cities, smart buildings (both homeand office), telecommunications, new generation media,smart grids, medical technology, collective and socialrobotics. Environment-focused applications include agri-culture, breeding, recycling, environment monitoring anddisaster alerting.
C. Cloud Computing
Cloud Computing [32,33] is a general term that refersto delivering computational services (servers, storage,databases, software, networking, analytics, etc.) overthe Internet («the cloud»). It is a model for enablingubiquitous, convenient, on-demand network access to ashared pool of configurable computing resources thatcan be rapidly provisioned and released with minimalmanagement effort or service provider interaction.
The NIST cloud model includes five basic character-istics, three service models and four deployment models.The following cloud characteristics are considered in[32]: 1) broad network access; 2) on-demand self-service;3) resource pooling; 4) rapid elasticity; 5) measuredservice.
Here broad network access means that various ca-pabilities are available over the network and accessedthrough standard mechanisms promoting the usage byheterogeneous thin or thick client platforms (e.g. mobilephones, tablets, laptops, and workstations).
In the context of on-demand self-servic, the consumercan unilaterally provision computing capabilities, such asserver time and network storage, as needed automaticallywithout requiring human interaction with each serviceprovider.
Moreover, the computing resources are pooled toserve multiple consumers by using a multi-tenant model,with different physical and virtual resources dynamicallyassigned and reassigned according to consumer demand.There is a sense of location independence in that thecustomer generally has no control or knowledge overthe exact location of the provided resources (storage,processing, memory, network bandwidth).
Besides, computing capabilities can be elastically pro-visioned and released, in some cases automatically, toscale rapidly outward and inward commensurate withdemand. To the consumer, the capabilities available for
191
provisioning often appear to be unlimited and can beappropriated in any quantity at any time.
Finally, resource usage can be monitored and reported,ensuring transparency for both the provider and con-sumer of the utilized service.
Three service models are: a) Software as a Service; b)Platform as a Service; c) Infrastructure as a Service. De-ployment models encompass private cloud, communitycloud, public cloud and hybrid cloud.
Such technologies as grid-computing, virtualization,service-oriented architectures (SOA) can be viewed aspredecessors of cloud computing. In particular, cloudcomputing extends SOA-applications.
Production resources and capacities can be intelli-gently sensed and connected into the cloud. The scalabil-ity of resources makes cloud computing interesting forbusiness owners, as it allows enterprises to start smallprojects and invest in more resources only if there arerises in further service demand.
Today most production-related companies require in-tensive data and knowledge sharing across sites andpartnerships. At the same time, the performance of cloudtechnologies will improve, achieving reaction times ofjust several milliseconds. As a result, machine dataand functionality will increasingly be deployed to thecloud, enabling more data-driven services for productionsystems.
D. Big Data and Their Analytics
The term «Big Data» stands for large data sets thatmay be analyzed computationally to reveal useful pat-terns, trends and associations. Ordinarily 3V Big Datamodel is used. Here we take a 5V concept of Big Data(Figure 6) that associates it with data volume, variety,velocity, veracity, value.
Figure 6. 5V Representation of Big Data
Here an immense data volume means such measure-ment units as Petabytes: 1 Petabyte = 1015 bytes, andhigher. Data variety is related to different distributed datasources, data velocity means the speed of data generationand processing, data veracity is attributed to a specificdata source and data value expresses its utility. Moreover,the concept of Big Data Value Chain has been introduced
in [34]; it encompasses big data capture, processing,interpretation (visualization), while preparing decision.
Now the main challenge in the field of Big Datais that the speed of data generation can exceed theprocessing capacity. In the near future, this situationcan seriously deteriorate. Indeed, IoT will be a majorsource of big data, contributing massive amounts ofstreamed information from billions of ICOs. Here M2Mcommunications will generate enormous Internet traffic.
In Industry 4.0 context, the collection and compre-hensive evaluation of data from many different sources– production equipment and systems, as well as productlifecycle and enterprise management– becomes a neces-sary step to support real-time decision making.
So decentralized Big Data Analytics and InformationMining are needed to cope with 5V. Specifically, VisualAnalytics supports analytical reasoning by interactiveuser-friendly visual interfaces. In manufacturing, DataAnalytics tools allow to optimize production quality, saveenergy and improve equipment.
Although many useful approaches and technologiesto cope with Big Data, such as MapReduce, Hadoop,Disco, have been successfully implemented, the need innew paradigms in Data Science becomes crucial. MiningInformation and Discovering Knowledge from Big Datarequires the support of special techniques and new ad-vanced technologies, in particular, data granulation andintelligent clustering methods and tools [35, 36].
E. Cybersecurity
With the increased connectivity and use of standardcommunications protocols that come with Industry 4.0,the need in cybersecuirity to protect critical industrialsystems and manufacturing lines from malware and cy-berattacks danger becomes crucial. Here the term mal-ware encompasses all types of cyberdangers, includingviruses, troyans, various spy programs, etc.
A well-known example of successful cyberattack wasthe use of Stuxnet virus against Iranian nuclear objects: itdeteriorated the operation of about 1000 centrifuges foruran concentration. This virus had unique characteristics:for the first time in the history of cyberattacks, virtualobject destroyed physical infrastructure.
In case of Internet of Things, the design, deploymentand maintenance of communications between heteroge-neous and geographically distributed things create grandchallenges related to security and privacy. To deal withsuch hard problems, reliable and safe communications,sophisticated identity and secure access management ofmachines and users are essential.
Today CPhS and IoT devices seem to remain vulnera-ble with respect to cyberattacks. Thus, the developmentof International Standards in the field of Cybersecurityremains a keynote task. These new standards can bebased on USA Federal Standard of Cyberrisks Manage-ment, NIST Special Publication 800-39, the Common
192
Vulnerability Scoring System, CVSS, the ISO/IEC31010Standard supporting risk multi-criteria analysis, etc.
F. Intelligent Simulation and Augmented Reality
The advent of the Industry 4.0 initiative has broughtserious changes to the simulation paradigm [37]. Nowit supposes modeling of manufacturing and other sys-tems by using virtual factory and digital twin concepts.The idea of digital twin extends the use of simulationtechniques to all phases of the product lifecycle, wherethe products are developed and thoroughly tested in avirtual environment. Combining the real life data withthe simulation results enables the rise of productivity andthe improvement of product quality. Thus, simulationswill be used more extensively in plant operations toleverage real-time data and mirror the physical world intoa virtual model, which can include machines, products,and humans.
The new simulation paradigm is closely related to con-siderable technological advances of augmented reality.The last one brings users the chance to experience anaugmented world by overlaying virtual information in thereal world. This way the user can be in touch with boththe real and virtual manufacturing worlds and receivereal-time data or statistics.
For Industry 4.0, this may bring several advantages. Itcan be the perfect method to represent relevant informa-tion for technicians and workers in the enterprise, allow-ing them to watch real time information from the workthey are performing. This is a suitable way to improvedecision-making procedures and working operations.
Augmented-reality-based systems support a variety ofservices, such as selecting parts in a warehouse andsending repair instructions over mobile devices. Anothergreat advantage is the possibility of enhancing industrialtraining and learning while reducing risks and costs.
In our opinion, an adequate extension of RAO intel-ligent simulation environment (see [38]) can provide agood solution for enabling intelligent simulation in anaugmented reality.
G. Additive Manufacturing
Additive manufacturing is a transformative approachto industrial production that enables the development oflighter, stronger parts and systems [1,25]. To differ fromusual manufacturing, additive manufacturing (AdM) addsmaterial to create an object. The processes of 3D print-ing and rapid prototyping are actually viewed as AdMcomponents. Methods and tools of AdM are widely usedin Industry 4.0 to produce small batches of customizedproducts. High-performance, decentralized AdM systemsreduce transport distances and stock on hand.
In perspective, a special attention will be paid tocombined approach based on «additive-subtractive» man-ufacturing [39].
H. Collaborative Robots
In the framework of Industry 4.0 robots becomemore autonomous, intelligent and cooperative. Here theconcepts of collective and collaborative robotics are ofspecial concern. Collective robotics considers variousgroups of robots working collectively to solve a problem.It investigates both teams of cognitive robots and swarmsof reactive robots [40,41]. In its turn, CollaborativeRobotics (briefly, Cobotics)[42] deals with cobotic sys-tems. A cobotic system is a «man – robot» system, wherethe participants collaborate in synergy to perform sometasks.
Collaborative robot is intended to physically interactwith humans in a shared workspace. This is the differ-ence with respect to conventional industrial robots, thatoperate autonomously or with a limited guidance.
In order to perform such a joint work hand by handwith human beings, any cobot needs to be equipped withpowerful onboard computer and complex sensor system,including an advanced computer vision and learningfacilities. It allows prevent the collisions of robot withhuman partners and obstacles, as well as operate in caseof software crash.
To differ from classical master-slave relations, human-robot partnership in cobotic systems is based on collabo-ration via interactive information management, where therobot partner can initiate the dialogue with human partnerto precise the task, request additional data or obtainhis evaluation of learning results. New opportunities forcobotic applications in Industry 4.0 are opened by astrategy of direct teaching «do as I do» by showing thenecessary motions to the robot.
Therefore, the main requirements for cobots are fo-cused on safety, light weight, flexibility, versatility, andcollaborative capacity.
Without the need of robot’s isolation, its integrationinto human workspace makes the cobotic system moreeconomical and productive, and opens up many newopportunities to compare with classical industrial robots.On the one hand, cobots increase information trans-parency via their ability to collect data and pass it on toother systems for analysis, modeling and so on. On theother hand, they provide technical assistance, in the sensethat they “physically support humans by conducting arange of tasks that are difficult, too exhausting, or unsafefor their human co-workers” [43].
According to ISO 10218, the following classes ofindustrial cobots can be viewed: a) robots-manipulatorssharing a workspace with humans (for instance, on theassembly line) to facilitate their workload (as a first inter-active industrial robot Baxter); b) mobile transportationrobots, as well as mobile robots working in productionrooms together with people; c) industrial multi-robotsystems. All these robots need the status of artificialcognitive and understanding agents.
193
IV. HOW TO BUILD ARTIFICIAL«UNDERSTANDING» AGENTS?
The development of Industry 4.0 supposes a totalenterprise agentification, where people, robots, indus-trial equipment (machines and materials), manufacturingsoftware tools and even enterprise products form anIntelligent Organization as a System of Multi-AgentSystems (MAS).
Most of these MAS must include cognitive, collabo-rative, «understanding» agents.
Let us recall some basic features of human cognition,which are of special interest for developers of artificialcognitive agents (also see [44,45]). Firstly, cognitionis an open system based on both available knowledgeand current data perception. Secondly, cognition doesnot make straight conclusions, but generates hypotheses,and these hypotheses should be confirmed or denied.Thirdly, agent cognition is intrinsically linked with theorganization of action (as information process, physicalmovement or local environment change). And fourthly,cognition is tightly connected with understanding. Onthe one hand, the cognitive capability itself and theresult of action strongly depend on the reached under-standing level (pre-understanding). On the other hand,human understanding is specified by cognitive capacities,available knowledge and language structure. Although,natural language understanding is driven not so muchby purely linguistic factors as by extra-linguistic factors,including personal experience and presupposition.
Understanding is a necessary condition for efficientcommunication between cognitive agents and their jointwork. It is obvious that the human-robot cooperation[42], development of Social Internet of Things [46]and Social Cyberphysical Systems [47,48] require somemutual understanding.
Understanding is not a new problem for AI, but earlierit was mainly considered in the context of natural lan-guage processing and text analysis. Such understandingobjects, as behavior, decisions, situations, remain almostunexplored.
Let us take the following basic definition from [44]:Understanding is a universal cognitive process (opera-tion) that evaluates an analyzed object (text, behavior,situation, phenomenon) on the basis of some standard,norm, pattern.
This definition has an axiological nature. It is foundedon value theory, because any evaluation implies somevalue (or logical inference from accepted values by usingsome general rules) [49]. Two basic operations to enableunderstanding are: a) the search for some norm andits formal representation; b) justification of the norm’sapplicability in a specific situation.
The level of agent task understanding can be specifiedby evaluating the results of his actions, which should notcontradict the norms of agent behavior.
Norms are social bans and constraints imposed on anagent by an organization (community). They represent aspecial case of evaluations: these are socially tested andfixed assessments.
The formal model of norm viewed as a prescription toaction is given by a quadruple:
NORM = 〈A, act,W,M〉.where A is a set of agents to whom a norm is
addressed, act ∈ ACT is an action being an object ofnormative regulation (the norm content), W is a set ofworlds, where the norm is useful (application conditionsor specific circumstances in which the action should beperformed or not), M is a set of basic modal systemsrelated to the action act, for example the system ofdeontic modalities MD = O,P, F. Here O stands for“obligatory”, P means “permitted” and F is “forbidden”.
An evaluation is transformed into a norm by somethreat of punishment, i.e. standardization of norms ismade through sanctions. Here a typical sanction rea-soning pattern is: q («obligatory q») and «if not q,then punishment or degradation». |par So an informationstructure of cognitive agent combines both descriptiveand normative models (Figure 7).
Figure 7. Two Sides of Agent Beliefs
Descriptions d contain the data on the states of en-vironment perceived by the agent, and prescriptions pgive the normative information about possible (permit-ted) actions or behavior patterns. Here a description ischaracterized by a truth value (description) T(d) and aprescription has a worth (or utility) value W(p). Hence,a truth value is the correspondence between the objectand its descriptive model (the object is primary), whereasan utility (worth) value gives the inverse mapping fromnormative model to this object (the norm is primary).A general agent understanding mechanism also has adualistic «Description-Evaluation» nature. Every objecthaving a standard prototype (pattern) is understandable,and the reason of misunderstanding consists in the lackof such pattern or its non-obviousness.
It is worth noticing that a pattern as a basis forunderstanding significantly differs from an example. Theexample refers to a real existing object, whereas the
194
pattern shows what should be done ideally. The examplesare taken to support descriptive models, but references tothe patterns and standards serve as justifications of normsand prescriptions.
This dualism is used in «Explanation-Understanding»relationships (Figure 8). Explanation, considered as thereduction of studied phenomenon to the scientific law,representative example or general truth, is based ona descriptive model and helps to understand it, butunderstanding as searching for rule or standard has anormative basis.
Figure 8. Relationships Between Explanation and Understanding
Logical lattices and bilattices of strong and weaknorms and anti-norms have been constructed to providean artificial agent of Industry 4.0 enterprise with somebasic understanding mechanisms.
Furthermore, in modeling artificial societies of In-dustry 4.0 it is worth employing earlier Russian theo-retical studies related to Technetics and Technocenosistheory [50,51]. The term «Technetics» stands for thetheory of technosphere evolution. A holistic approachto techniques, technologies, materials, products, waste istaken. An important part of technetics is technogeneticsthat encompasses the problems of creation and transferof hereditary information by design and technologicaldocumentation and other means.
V. CONCLUSIONTo design user-oriented intelligent systems for Indus-
try 4.0, in particular, agent-based, multi-agent systemsand artificial societies, we have to integrate usual se-mantic technologies with open pragmatic technologies,for instance, Peirce’s logical pragmatics and modernpragmatic logics (see [45,49]). We need new theoreticalmethods and models in representing agent’s pragmatics(pragmatic worlds and spaces, ontological pragmatics),modeling such pragmatic concepts as beliefs, evaluationsand norms, synthesizing pragmatics-based logical andlogical-semiotic systems. Here the principle «First prag-matics, then calculus» has to be satisfied. The synergyof semantic and pragmatic technologies is a necessarycondition for building advanced intelligent agents.
In our opinion, there exist at least three different waysin developing new generation technologies for Industry4.0. The first one consists in building intelligent coun-terparts of main Industry 4.0 components by employing
conventional AI technologies, such as: Intelligent Simu-lation [38], Intelligent Cyberphysical Systems, IntelligentCloud Computing and so on. The second way supposes«putting the old wine in new bottles», for example, thereturn back to technocenosis and populations of artificialagents. Finally we have to develop some new trends in AIand Cognitive Sciences, such as General UnderstandingTheory, Granular Measurements by Cognitive SensorNetworks, Context Aware Search, and so on. The eraof artificial-agent-based Industry 4.0 just begins.
VI. ACKNOWLEDGEMENT
The research work is supported by the State Order(Goszadaniye) No 2.7918.2017 and Russian Foundationfor Basic Research, Project No 19-07-01208.
REFERENCES
[1] H.Kagermann, J.Helbig, A.Hellinger, W.Wahlster, Recommen-dations for Implementing the Strategic Initiative Industrie 4.0:Securing the Future of German Manufacturing Industry. FinalReport of the Industrie 4.0 Working Group.
[2] K.Schwab, The Fourth Industrial Revolution. Geneva, WorldEconomic Forum, 2016, 172 p.
[3] Industry 4.0: Building the Digital Enterprise. Availableat: https://www.pwc.com/gx/en/industries/industries-4.0/landing-page/industry-4.0-building-your-digital-enterprise-april-2016.pdf
[4] K.D. Thoben, S. Wiesner, T. Wuest, Industrie 4.0 and SmartManufacturing – A Review of Research Issues and ApplicationExamples. International Journal of Automation and Technology,vol.11, No 1, 2017, pp. 4-16.
[5] M.Skilton M., F.Howsepian, The Fourth Industrial Revolution:Responding to the Impact of Artificial Intelligence on Business.London, Palgrave Macmillan, 2018
[6] V.V.Golenkov, N.A.Guliakina, The Project of Open SemanticTechnology for Component Design of Intelligent Systems. Part1. Principles of Designing. Ontology of Designing, 2014, No1,pp.42-64 (in Russian).
[7] V.V.Golenkov, N.A.Guliakina, I.Davydenko, Methods and Toolsfor Interoperability of Computer Systems. Open Semantic Tech-nologies for Intelligent Systems, Minsk, BSUIR, 2019, Iss. 3 (inRussian).
[8] Ch.Peirce, Collected Papers, vol.1-6.Cambridge MA:HarvardUniversity Press, 1965-1967.
[9] V.V.Martynov, Fundamentals of Semantic Coding. Essay ofKnowledge Representation and Transformation. Minsk, EuropeanHumanitarian University, 2001 (in Russian).
[10] V.B.Tarassov, From Semantic Code to Cognitive Linguistics,Semiotics and Informatics: On Martynov’s Scientific Heritage.Proceedings of the IVth International Conference "Open SemanticTechnologies for Intelligent Systems" , Minsk, BSUIR, 2014,pp.39-48 (in Russian).
[11] M.C.Roco, W.S.Bainbridge (Eds.) Converging Technologies forImproving Human Performance. NSF/DOC-sponsored report,Heidelberg, Springer-Verlag, 2003.
[12] Ch. Roser, Industrial Revolutions and Future View. Available at:http://www.allaboutlean.com
[13] V.F.Gornev, Problems and Technologies of Complex Automation.Computer Aided Design, part1, No 4, pp.41-44; part 2, No 1,1999, pp.28-35 (in Russian).
[14] F.Kimura, H.Suzuki, Product Life Cycle Modeling for InverseManufacturing. Proceedings of IFIP WG 5.3 International Con-ference on Life Cycle Modeling for Innovative Products andProcesses (PROLAMAT’ 95, November 29-December 1, 1995),F.L. Krause, H. Hansen (Eds.), Berlin, Springer Verlag, 1996,pp.81-89.
[15] F.B. Vernadat, Enterprise Modeling and Integration: Principlesand Applications. London, Chapman and Hall, 1996.
195
[16] V.B. Tarassov, The Concept of Meta-CIM: from CIM to En-terprise Internet/Intranet-Networks, Software and Systems, 1998,No3, pp.19-22 (in Russian).
[17] S.L. Goldman, R.N. Nagel, K. Preiss, Agile Competitors andVirtual Organizations: Strategies for Enriching the Customer. NewYork, Van Nostrand Reinhold, 1995.
[18] M.Hardwick, D.Spooner, T.Rando, K.Morris, Sharing Manufac-turing Information in Virtual Enterprises. Communications of theACM,1996, vol.39, No 2, pp.46-54.
[19] V.V.Emelyanov, V.M.Kureichik, V.V.Kureichik, Theory and Prac-tice of Evolutionary Modeling. Moscow, PhysMathLit, 2003 (inRussian).
[20] V.F. Gornev, V.B. Tarassov, On Some Projects and Initiatives ofthe International Program on IMS. Computer Aided Design, 1999,No 2, pp.42-46 (in Russian).
[21] R.H.Brown,H.Syntera, Global Manufacturing in the 21st Cen-trury”. Final Report. IMS, 1999.
[22] TOVE Ontology Project 2010. Available at:http://www.eil.utoronto.ca/enterprise-modelling.
[23] C.Baur, D.Wee, Manufacturing’s Next Act. McKinsey & Com-pany, June 2015.
[24] S.C. Suh, U.J. Tanik, J.N. Carbone, A.E. Eroglu (Eds.), AppliedCyber-Physical Systems. Heidelberg, Springer-Verlag, 2014.
[25] Ph. Gerbert, M. Lorenz, M. Rusmann, et al. Industry 4.0: TheFuture of Productivity and Growth in Manufacturing Industries.BCG, April 2015.
[26] J.L.G.Dietz et al. Enterprise Engineering Manifesto, Advances inEnterprise Engineering I, Lecture Notes in Business InformationProcessing,2011, Vol.10.
[27] Yu.F. Telnov, Enterprise Engineering Based on Intelligent Tech-nologies. Information-Measurement and Control Systems, 2013,No 11, part 2, pp.55-60 (in Russian).
[28] H. Sundmaeker, P. Guillemin, P. Friess, S. Woelffle (Eds.), Visionand Challenges for Realizing the Internet of Things. Luxembourg,EU, CERP IoT, 2010.
[29] K.Ashton, That Internet of Things, RFID Journal, 22 June 2009.[30] Gartner, Internet of Things. Available at:
http://www.gartner.com/it-glossary/internet-of-things/[31] D.Uckelmann, M.Harrison, F.Michahelles, (Eds.) Architecting the
Internet of Things. Heidelberg, Springer-Verlag, 2011.[32] P.Mell, T.Grance, The NIST Definition of Cloud Computing
Gaithersburg: National Institute of Standards and Technology,2011.
[33] N.K.Sehgal, P.Ch.Bhatt, Cloud Computing: Concepts and Prac-tices. Heidelberg, Springer-Verlag, 2018.
[34] H.J.Miller, P.Mork, From Data to Decisions: a Value Chain forBig Data. IT Prof,2013, vol.15, pp.57-59.
[35] L.Zadeh, Toward a Theory of Fuzzy Information Granulation andits Centrality in Human Reasoning and Fuzzy Logic. Fuzzy Setsand Systems, 1997, vol.90, pp.111-127.
[36] W.Pedrycz, Granular Computing for Data Analytics: a Manifestoof Human-Centric Computing.Soft Measurements and Comput-ing, 2018, vol.2, No 7, pp.4-21.
[37] B.Rodic, Industry 4.0 and the New Simulation ModelingParadigm. Organizacija, vol.50, No 3, 2017, pp.193-207.
[38] A.Artiba, V.V.Emelyanov, S.I.Iassinovski, Introduction to Intelli-gent Simulation. RAO Language. Dordrecht, Kluwer AcademicPublishers, 1998.
[39] G.B.Evguenev, Methods of Programming Combined Additive-Subtracting Processing. Proceedings of Higher Education Insti-tutions: Machine Building, 2017, No 4, pp.47-56 (in Russian).
[40] S.Kernbach (Ed.) Handbook of Collective Robotics: Fundamen-tals and Challenges. Singapore, Pan Stanford Publ. Pte. Ltd.,2013.
[41] V.E.Karpov, I.P.Karpova, A.A.Kulinich, Societies of Robots.Moscow, Lenand, 2019 (in Russian).
[42] M.Peshkin., J.Colgate, Cobots. Industrial Robots, 1999, vol.26,No5, pp. 335-341.
[43] J.Faneuff, Designing for Collaborative Robotics. Sebastopol Ca,O’Reilly, 2016.
[44] V.B.Tarassov, Problem of Understanding: the Present and Futureof Artificial Intelligence. Proceedings of the 5th International
Conference "Open Semantic Technologies for Intelligent Systems’,Minsk, BSUIR, 2015, pp. 25-42 (in Russian).
[45] V.B.Tarassov, Information Granulation, Cognitive Logic and Nat-ural Pragmatics for Intelligent Agents.Open Semantic Technolo-gies for Intelligent Systems, 2018, Issue 2, pp.45-54.
[46] L.Atzori, A.Iera, G.Morabito, M.Nitti, The Social Internet ofThings – When Social Networks Meet the Internet of Things:Concept, Architecture and Network Characterization. ComputerNetworks, 2012, vol.56, No 16, pp.3594–3608.
[47] F.Dressler, Cyber Physical Social Systems: Towards Deeply Inte-grated Hybridized Systems. Proceedings of 2018 InternationalConference on Computing, Networking and Communications,2018, pp.420-424.
[48] E.M.Frazzon, J.Hartmann, Th.Makuschewitz, B.Scholtz, TowardsSocio-Cyber-Physical Systems in Production Networks. ProcediaCIRP, 2013, vol. 7, pp.49-54.
[49] A.A.Iwin, Logic of Evaluations and Norms: Philosophical,Methodological and Applied Aspects. Moscow, Prospect, 2016(in Russian).
[50] B.I. Kudrin, Introduction to Technetics. Tomsk, TGU Press, 1993(in Russian).
[51] D.A.Pospelov, V.I.Varshavsky, The Orchestra Plays without aConductor. Reflections on Evolution and Control of Some Tech-nical Systems. Moscow, Librokom, 2009 (in Russian).
CЛЕДУЮЩАЯ СТАДИЯ ИНДУСТРИИ 4.0: ОТКОГНИТИВНЫХ К КОЛЛАБОРАТИВНЫМИ
«ПОНИМАЮЩИМ» АГЕНТАМ
Тарасов В. Б.ФГБОУ ВО «Московский государственныйтехнический университет им. Н.Э.Баумана»
В работе показана роль искусственных когнитивных, кол-лаборативных и «понимающих» агентов в развитии техно-логий Индустрии 4.0. Обоснована целесообразность инте-грации открытых семантических и прагматических техно-логий в русле интеллектуализации Индустрии 4.0. Предва-рительно изложены основные характеристики и технологиипредыдущих промышленных революций. Особое вниманиеуделено эволюции производственных систем в XX-м веке.Проанализированы темы и основные результаты первоймеждународной программы IMS по интеллектуальным про-изводственным системам. В основной части статьи пред-ставлены главные идеи и принципы стратегии Индустрия4.0. Ее сердцевина – киберфизические системы, которыеобеспечивают единство физического и виртуального мировна производстве. Описано семейство базовых технологий исредств Индустрии 4.0, которое включает: технологии инжи-ниринга и интеграции предприятий; интернет вещей; облач-ные технологии; большие данные и средства их аналитики;имитационное моделирование; виртуальную и дополненнуюреальность; аддитивные технологии; автономные и коллабо-ративные роботы; средства обеспечения кибербезопасности.Выдвинут тезис о проведении сквозной агентификации пред-приятий Индустрии 4.0, согласно которому они понимаютсякак смешанные сообщества естественных и искусственныхагентов. В заключительной части статьи рассмотрена зада-ча построения искусственных «понимающих» агентов, длярешения которой использованы аксиологический подход ипрагматические логики.
Received 21.12.18
196
Application of fuzzy models of evolutionarydevelopment in optimal control of the system
of planned preventative maintenance and repairof equipment for multistage production
Boris V. Paliukh, Alexander N. Vetrov, Irina A. Egereva, Irina I. EmelyanovaTver State Technical University
Tver, [email protected], [email protected], [email protected], [email protected]
Abstract—The paper presents a mathematical model ofcontrol of planned preventative maintenance and repairof equipment under fuzzy conditions, when the state ofthe repairable unit in question may deviate from the onespecified by the technical documentation. Introduction ofan informational system for performing repair and rou-tine preventative maintenance works together with RFIDtechnology ensure efficient control of one of the crucialcomponents of production, that is, of maintenance andrepair system. The suggested control model is intendedfor supporting quick making of management decisionsin emergencies. One can see the result of the suggestedmodel implementation by the example of a paper industryenterprise.
Keywords—fuzzy models, optimal control, planned pre-ventative maintenance and repair of equipment, RFIDtechnology
I. INTRODUCTION
When studying industrial production economic efficiencyand also evaluating opportunities for the enterprise growthand development, special attention is paid to technical andeconomic indices showing the production equipment perfor-mance level [1,2]. These indices include equipment downtimeunder repair per a repairable unit, repair prime cost per arepairable unit, the number of breakdowns and unplannedrepairs per equipment unit, maintenance and repair manpowerlabour efficiency, etc.
Maintenance and repair unit of the enterprise performsequipment maintenance and repair intended to keep the equip-ment in constant working order. Achieving this goal in the mostcost-saving way implies minimization of the total costs causedby the equipment breakdown and keeping this equipment ingood working order.
Planned preventative maintenance and repair system impliesperforming preventative works for equipment maintenance andplanned repairs every certain number of equipment operationhours; the sequence and schedule of these works are determinedby the special features of the equipment and its operationenvironment. Besides following the set maintenance and repairstandards (such as repair cycles and their structure, repaircomplexity categories, repair works labour input and materialsconsumption together with mate-rial inventories for the repairneeds), maintenance and repair unit staff must be able to applymodern means of equipment status diagnostics, implement
automatization of equipment maintenance and repair, introduceinnovative technological equipment and apply new efficientmethods of maintenance and repair planning and management,etc.
The production equipment full working capacity being cru-cial for the enterprise general operation and successful develop-ment, the paper suggests creating a model of optimal control ofa fuzzy continuous system of planned preventative maintenanceand repair of the said equipment.
In the management process, it is necessary to operate withqualitative in-formation, which is the main cause of uncertaintyand explains the use of the concept of fuzziness in this paper[3,4,12]. The problem of managing fuzzy systems is relevantand is the subject of research into solving applied problems[eg, 5,6].
II. TASK ASSIGNMENT FOR CONTROLLING THESYSTEM OF PLANNED PREVENTATIVE MAINTENANCE
AND REPAIR
At industrial enterprises, equipment is divided into severalgroups, each of which includes several units, which, in theirturn, consist of tens or hundreds of elements.
For instance, according to paper production technology, thereare 9 groups of equipment [7]: for raw material preparation M1,for pulping M2, for pulp receiving, rinsing, sorting, thickeningand bleaching M3, for chemicals preparation and regenerationM4, for wood pulp production M5, for paper pulp preparationM6, for paper and cardboard production M7, for marketablecellulose production M8, for paper and cardboard finishing,cutting, sorting and packing M9.
According to maintenance and repair schedule, each produc-tion equipment element undergoes maintenance, which consistsof routine interrepair maintenance and periodical preventativerepair operations, and also of planned maintenance and repair,which, in its turn, includes routine and full maintenance andrepair.
Considering control of planned preventative maintenanceand repair (PPMR) system, one should take into accountcontrol system fuzziness, as equipment may break down duringoperation due to heavy workload, defects, insufficient qualityof the previously conducted maintenance and repair, and so on.
Then it is necessary to make quick decisions on emergencyrepair or replacement of equipment component parts, on mod-ernization, on searching for alternative component parts, etc.,when the data, goals and limitations are too complex andunclear [8].
197
Let us consider the enterprise production system σ as anaggregation of fuzzy systems with multistep control UN , whereSN is a step of controlling element i of subsystem Mn,N is the number of a control step for element n,M is a group of equipment,i is the number of an element in group M , i = [1, n],XN is the subsystem status achieved by control UN ,UN – space of controls of subsystem M .
Figure 1. Monitoring of Equipment Group.
Let us suppose that at step N = 1 element n of equipmentgroup M is put in-to operation, at N = 2 the element isundergoing preventative maintenance, at N = 3 planned main-tenance and repair are in progress, and so on up to final step N ,when the equipment is beyond repair (for instance, the servicelife is over). Besides the stages of the planned preventativeworks which are set by the regulations, the equipment controlsubsystem may experience emergencies.
Therewith, the task of the production technological systemoptimal control is to keep the enterprise production equipmentin good working order.
Management decisions aimed at improving productionequipment operation may include personnel training, mainte-nance process control, checking the inventories of componentparts and repairable units, balancing the work-load of main-tenance and repair workshops, reducing maintenance time byapplying innovative technologies, reducing human error influ-ence, reducing error probability during equipment operation,and monitoring the funds allocated for supporting operationreliability of the production equipment.
III. MODELING A FUZZY SYSTEM USING FUZZYRELATIONS
Let us demonstrate a generalized formal mathematical modelof control for a single production unit under conditions offuzziness. From now on, when considering the control task,we will use one modification of fuzzy relations composition.
Let X , Y and Z be certain sets. Let us assume that at aX×Y set a fuzzy relation A with membership function µA isdefined and at a Z×Y set a fuzzy relation B with membershipfunction µB is defined. Therefore, the A B composition offuzzy set A and B is the fuzzy relation in X × Z space withthe membership function
µAB(x, z) = supy∈Y
min[µA(x, y), µB(y, z)] (1)
(see eg. [3]).Let’s now assume that in X space the fuzzy set R with
membership function µR is defined. Therefore, the fuzzyrelation µA induces the fuzzy set R A in the Y space. Inaccordance with (1) the membership function µRA of R Aset is given by an equation
µRA(y) = supx∈X
min[µR(x), µA(x, y)].
Let us assume that at X set the fuzzy relation S withmembership function µS is defined. Further, let us assume thatin X space G set with membership function µG is also defined.Therefore we can determine the S G composition of fuzzysets S and G and following (1) the membership function µSGof S G set will be defined by equation
µSG(x1) = supx2∈X
min[µS(x1, x2), µG(x2)]. (2)
One can readily see, for each x1 ∈ X the membershipdegree of µSG(x1) of x1 the fuzzy set G is defined byequation (2).
It is equation (2) that we will further consider as a modelof a fuzzy system.
IV. STATES EVOLUTION CONTROL
Let X and U be certain compact metric spaces. Let usconsider H is control system when X is state space and Uis control space.
Let’s assume that evolution of H system state is characterizedby the fuzzy relation S representing fuzzy set S in X×U×Xpace with membership function µS , provided that the initialstate x0 ∈ X is defined.
As a result of choosing of u0 ∈ U control the system goesinto some new state x1 which was earlier unknown. It is onlyknown that with u0 and x0 fixed, x0, u0, and x1 variablesare related by the fuzzy relation S with membership functionµS(x0, u0, x1). In other words, with u0 and x0 fixed at pointof time n = 0 the state x1 can be defined only by value ofmembership function µS(x0, u0, x1). However, at point of timen = 1 we can observe exact value of state x1.
Let us consider that the control aim is characterized by fuzzygoal set G in X space with membership function µG. Let usalso assume that both functions µS and µG are continuous inthe range of their definition.
Now let’s assume that time N of end of system work isdefined. The control problem is to search the sequence
u0, u1, ..., uN−1 (3)
of points of U set maximizing the membership degree of x0states to fuzzy set G with fuzzy relations with membershipfunctions
µS(x0, u0, x1), µS(x1, u1, x2), ...,
µS(xN−1, uN−1, xN ).
Therefore, the fuzzy set G is the control aim and the problemconsists in searching the control sequences (3) providing themaximal membership degree of the state x0 to the fuzzy setG with which the evolution of system state is described as thecomposition of fuzzy sets S and G. By equation
DN = S ... S G︸ ︷︷ ︸N
198
let’s put for consideration the fuzzy set DN being conditionalfor variables (3) in the X space with membership function µDN
satisfying the equation
µDN (x0 | u0, u1, ..., uN−1) = maxx1,x2,...,xN
min[µS(x0, u0, x1), µS(x1, u1, x2), ...,
µS(xN−1, uN−1, xN ), µG(xN ].
Therefore according to equation (2) µDN (x0 |u0, u1, ..., uN−1) the values of function µDN have theform of the membership degree of the state x0 to G set withthe use of any fixed sequence of control of (3) kind. Let us set
uN (x0) = maxu0,u1,...,uN−1
µDN (x0 | u0, u1, ..., uN−1). (4)
Following [9], let us consider the initial task in the contextof task family where x0 and N are variable values. Therefore,with N = 0 the required membership degree x0 to G set withthe fuzzy relation S is described by the equation
u0(x0) = uG(x0). (5)
Function µ0 is continuous by convention over all of theintervals at X set. Moreover because of continuity of functionsit is easy to note that for each function f which is defined andcontinuous over all of the intervals at X and possesses valuesat the interval [0, 1], the function
g(x0, u0, x1) = min[µS(x0, u0, x1), f(x1)]
is continuous over all of the intervals. But X and U spaces arecompact. Therefore, the function
h(x0) = supu0,x1
min[µS(x0, u0, x1), f(x1)] =
= maxu0,x1
min[µS(x0, u0, x1), f(x1)]
is continuous over all of the intervals at X set. Provided that
maxu0,u1,...,uN
µDN+1(x0 | u0, u1, ..., uN ) =
= maxu0,x1
min[µS(x0, u0, x1), maxu0,u1,...,uN
µDN (x1 | u1, u2, ..., uN )]
(see, eg. [9]).Then by virtue of (4) for certain N the equation
uN+1(x0) = maxu0,u1
min[µS(x0, u0, x1), uN (x1) (6)
is executed where uN+1(x0) is the maximal membershipdegree of the state x0 to the G set with the relation S and thecondition where end of system work time is equal to N + 1,and uN (x1) is the maximal membership degree of the state x1to the G set with relation S and the condition where end ofsystem work time is equal to N .
One can readily see that recurrence relationship (6) with thecondition (5) is similar to Bellman’s functional equation forclassical problems of dynamic programming. This relationshipinterprets the control u0 as function of time N and the statex0, i.e.
u0 = u∗0(x0, N), N = 1, 2, ... . (7)
Now let us note that for each N the function µN+1 is defined.In addition, if in equations
minx0∈X
µG(x0) > 0
andmin
x0,u0,x1
µS(x0, u0, x1) > 0
are executed, then for all N = 0, 1, 2, . . . the in equation
µN (x0) > 0
is correct.It is obvious that in this case we can always imply a well-
defined task.
V. IMPLEMENTATION OF THE SYSTEM OF OPTIMALCONTROL OF THE SYSTEM OF PLANNED
PREVENTATIVE MAINTENANCE AND REPAIR OFPRODUCTION EQUIPMENT EXEMPLIFIED BY A
VERTICAL-TYPE HYDRAULIC PULPER
A hydraulic pulper is a machine for pulping waste andbroken paper with a rotating bladed disk located at the bottomor at the side of a cylindrical tank. Around the disk there arestationary blades and a sieve for extracting the pulp [4].
According to the technical documentation, hydraulic pulperpreventative check-up is done during the idle periods. ThePPMR system sets the hydraulic pulper interrepair cycle of17t hours. This period includes four sessions of routine main-tenance and repair 8 hours each, 1 medium maintenance andre-pair lasting 24 hours and 1 full maintenance and repair thattakes 72 hours.
In order both to ensure quickest possible handling of emer-gencies and to check whether preventative maintenance andrepair works are performed by the maintenance and repairunit staff in full scope and in due time, the enterprise chiefmechanic decided to introduce an informational system ofrepairs prevention, which includes a database keeping theinformation on all control stages for every equipment unit.
Furthermore, each individual equipment unit is identifiedwith RFID (Radio Frequency Identification) technology, whichimplies using an RFID-tag and an RFID-reader [10-11]. AnRFID-tag contains unique tag number and transfers data tothe RFID-reader, which registers data transmission, reads theinformation from the tag and transfers it to the informationalsystem.
Further on, during equipment operation (including check-ups on keeping to regulations and standards of performingmaintenance and repair works), a staff member can use a tabletto see all the necessary information on the equipment unit inquestion (fig. 2 – the data the staff member sees on the tabletscreen).
For instance, the screen document template (fig.2) containsthe information on “hydraulic pulper” equipment with ID(number), shows the data on performing routine maintenanceand repair, preventative checks-ups, planned and full mainte-nance and repair. The “Comments” field says what elementsneed special attention during further maintenance; in case theequipment unit suffered a failure, the field states the failurecause and the ways to avoid experiencing this emergency onceagain.
Application of the chosen control method results in reducingequipment downtime under repair together with the number ofbreakdowns, failures and unscheduled maintenance and repairsper equipment unit; maintenance and repair prime cost is alsoreduced by performing timely preventative works.
VI. CONCLUSIONS
The suggested model of control of the system of plannedpreventative maintenance and repair describes control of tech-nological units of equipment at an enterprise under conditionsof continuous production.
199
Figure 2. Example of software implementation of the model ofmaintenance and repair system control under conditions of fuzziness.
Application of this model reduces the time of makingdecisions in emergencies and also facilitates efficient man-agement of planned and routine maintenance and repair ofthe equipment. The mathematical model of control of plannedpreventative maintenance and repair under fuzzy conditionsis applicable at any industrial enterprise having continuousproduction cycle.
VII. ACKNOWLEDGEMENTS
The research was supported by the Russian Foundation forBasic Research (project 17-07-01339).
REFERENCES
[1] Romanenko I. V. Ekonomika predpriyatiya [Economics of theenterprise]. Moscow, Finansy i statistika, 2002. 208 p.
[2] Gordienko G. Usloviya strukturnogo i tekhnicheskogo ob-novleniya [Conditions of structural and technical renovation].Ekonomist [Economist], 2002, No 8, pp. 20-28.
[3] L. Zadeh. Ponyatie lingvisticheskoi peremennoi i ego primeneniek prinyatiyu priblizhennykh reshenii [The concept of linguisticvariable and its application to adoption of the approximatesolutions]. Moscow, Mir, 1973. 167 p.
[4] S. A. Orlovsky. Problemy prinyatiya reshenii pri nechetkoiiskhodnoi informatsii [Decision-making problems with fuzzyinitial information]. Moscow, Nauka, 1981. 208 p.
[5] S. M. Dzyuba, B. V. Paluch, I. A. Egereva. Optimal’noe upravle-nie nechetkimi mnogostadiinymi protsessami [On the optimalcontrol of fuzzy multistage processes]. XII Vserossiiskaya kon-ferentsiya po problemam upravleniya [XII all-Russian conferenceon control problems], 2014, pp. 3968-3972.
[6] Shcherbatov I.A. Upravlenie slozhnymi slaboformalizuemymimnogokomponentnymi sistemami [Managing complex, weaklyformalizable multi-component systems]. Rostov-na-Donu, YuNTsRAN, 2015. 268 p.
[7] R 50-54-25-87 Rekomendatsii. Oborudovanie dlya proizvodstvatsellyulozy, bumagi i kartona. Terminy i opredeleniya [R 50-54-25-87 Recommendations. Equipment for the production ofpulp, paper and card-board. Terms and Definitions]. Moscow,VNIINMASH, 1987. 30p.
[8] GOST 21623-76 Sistema tekhnicheskogo obsluzhivaniya i re-monta tekhniki. Pokazateli dlya otsenki remontoprigodnosti. Ter-miny i opredeleniya [State Standart R-21623-76. System oftechnical maintenance and repair of equipment. Characteristicsfor evaluation of maintainability and repairability. Terms anddefinitions]. Moscow, Standartinform, 2006. 14 p.
[9] A.P. Afanas’ev, and S. M. Dzyuba. Ustoichivost’ po Puassonu vdinamicheskikh i nepreryvnykh periodicheskikh sistemakh [Pois-son Stability in Dynamical and Continuous Periodic Systems].Moscow, LKI, 2007. 240 p.
[10] Hasan N. Roadmap for RFID Implementation in Libraries: Issuesand Challenges. Int. J. Info. Libr. Soc, 2014, No 3(1), pp. 65–71.
[11] Grigoriev P. V. Osobennosti tekhnologii RFID i ee primenenie[Features of RFID technology and its application]. Molodoiuchenyi [Young scientist], 2016, No 11, p. 317-322.
[12] R. Bellman and L. Zadeh. Prinyatie reshenii v rasplyvchatykhusloviyakh. Voprosy analiza i protsedury prinyatiya reshenii[Decision-making in vague terms. Questions analysis anddecision-making procedures]. Moscow, Mir, 1976. 46 p.
ОПТИМАЛЬНОЕ УПРАВЛЕНИЕ СИСТЕМОЙПЛАНОВО-ПРЕДУПРЕДИТЕЛЬНОГО
РЕМОНТА ОБОРУДОВАНИЯМНОГОСТАДИЙНОГО ПРОИЗВОДСТВА СПРИМЕНЕНИЕМ НЕЧЕТКИХМОДЕЛЕЙ
ЭВОЛЮЦИОННОГО РАЗВИТИЯ
Палюх Б.В., Ветров А.Н.,Егерева И.А., Емельянова И.И.
В работе представлена математическая модель управле-ния системой планово-предупредительного ремонта обору-дования в условиях нечеткости, когда состояние рассмат-риваемой ремонтной единицы может иметь отклонение отсостояния, предусмотренного технической документацией.
Внедрение информационной системы по проведению ре-монтных и текущих профилактических работ в совокупно-сти с технологией RFID позволяют эффективно управлятьодной из важнейших составляющихпроизводства – системойтехнического обслуживания и ремонта.
Предложенная модель управления направлена на под-держку быстрого принятия управленческих решений приаварийных и внештатных ситуациях. Приведен результатприменения предложенной модели на примере предприятиябумажной промышленности.
Received 10.12.18
200
Implementation of intelligent forecastingsubsystem of real-time
Alexander Eremeev, Alexander KozhukhovInstitute of Automatics and Computer Engineering
Moscow Power Engineering InstituteMoscow, Russia
[email protected], [email protected]
Natalia GuliakinaBelarussian State University
Informatics and RadioelectronicsMinsk, Belarus
Abstract—The paper describes architecture of intelli-gent forecasting subsystem of real-time based on multi-agent temporal differences reinforcement learning, statisti-cal module and monitoring module with milestone anytimealgorithm. Analysis of anytime algorithms were made interms of using into the forecasting subsystem type ofintelligent decision support system of real-time for im-proving performance and reducing response and executiontime. The considered tools can be used to implementthe possibility of self-learning and adaptation both in theintelligent systems of real-time created on their basis, andin the actual tools for creating such systems. The work wassupported by BRFFR projects 17-07-00553 a, 18-51-00007Bel a, F16R-102.
Keywords—artificial intelligence, intelligent system, realtime, reinforcement learning, forecasting, decision support,anytime algorithm.
I. INTRODUCTION
Reinforcement learning (RL) methods [1], based onthe using large amount of in-formation for learning inarbitrary environment, are the most rapidly developingareas of artificial intelligence, related with the develop-ment of advanced intelligent systems of real-time (ISRT) typical example of which is an intelligent decisionsupport system of real-time (IDSS RT) [2].
One of the most promising in terms of use in IDSS RTand central in RL is Temporal Difference (TD) learning[1]. TD-learning process is based directly on experiencewith TD-error, bootstrapping, in a model-free, online,and fully incremental way. Therefore, process do notrequire knowledge of the environment model with itsrewards and the probability distribution of the next states.The fact that TD-methods are adaptive is very importantfor the IS of semiotic type able to adapt to changes inthe controlled object and environment [3].
Using the multi-agent approach contains of groupsof autonomous interacting entities (agents) having acommon integration environment and capable to receive,store, process and transmit information in order to ad-dress their own and corporate (common to the groupof agents) analysis tasks and synthesis information isthe fastest growing and promising approach for dynamicdistributed control systems and data mining systems,
including IDSS RT. Multi-agent systems could be charac-terized by the possibility of parallel computing, exchangeof experience between the agents, resiliency, scalability,etc. [4].
Usually data encountered by an online RL-agent isnon-stationary, and online RL updates are strongly corre-lated. Deep reinforcement learning (DRL) approach pro-vide rich representations that can enable RL-algorithmsto perform effectively and enables automatic featureengineering and end-to-end learning through gradientdescent, so that reliance on environment is significantlyreduced or even removed. Common idea of DRL isstoring the agent’s data in an experience replay memorywhere the data can be batched or randomly sampled fromdifferent time-steps. Aggregating over memory in thisway reduces non-stationarity and decorrelates updates,but at the same time limits the methods to off-policyRL-algorithms [5, 6, 7].
When modern IDSS RT are developing, importantconsideration should be given to means of forecasting thesituation at the object, consequences of decisions, expertmethods and learning tools [8]. In addition, attentionshould be given to optimal using of system availableresources and ability to work in the environment with re-strictions in time. These resources are necessary for mod-ification and adaptation of IDSS RT regarding changesin object and external environment and for enhancing theapplication field and improving system performance.
For solving these problems, developed forecastingsubsystem using parallel algorithms for deep reinforce-ment learning, statistical methods and monitoring sub-module using milestone anytime algorithm that can re-ceive acceptable information within the resources andtime constraints were considered.
II. ANALYSIS OF ANYTIME ALGORITHMS
When agent interacting in complex dynamic real-timesystem, where available time for planning is highlylimited, generating optimal solutions can be infeasible.In such situations, the agent must be satisfied with thebest solution that can be generate within the available
201
computation time and within the range of tolerance oferror. A useful set of algorithms for generating suchsolutions are known as anytime algorithms. Typically,these start out by computing an initial, potentially highlysub-optimal solution and then improve this solution astime allows [9].
Anytime algorithms allow making a tradeoff betweencomputation time and solution quality, making it possibleto compute approximate solutions to complex problemsunder time constraints. They also need to have settingsthat can adjust the flexibility of finding a tradeoff.They can be represented as sampling rates or iterativeimprovement functions that affect the quality in termsof accuracy, coverage, certainty and level of detail. Atradeoff between quality and time can be achieve byseveral methods:
• milestone method that executed in the minimumperiod of time and made subsequent evaluationof progress at control points. Based on the re-maining time, the algorithm can decide to performboth mandatory and optional operations or simplymandatory operations;
• sieve functions that allows to skip the calculationsteps. So, the minimum useful selection can bereached in a shorter period of time;
• multiple versions of the same algorithm in whichintensive calculations can be replaced with fasterbut less accurate versions of the same algorithm;
For each of the above methods, it is necessary that thedifferent implementation approaches have the ability tomeasure the quality of explicit metrics in the currentstate. At any time, the way to execute the algorithmdepends on several factors such as: the quality of theavailable solution, the prospects for further improvingthe quality of the solution, current time, cost of delayingthe actions taken, current state of the environment andprospects for further environmental change that couldbe deter-mined only in runtime. Thus, it is necessary todetermine the path that provides the most optimal resultrelative to the current state of the environment.
Anytime algorithms should be able to be interrupt atany time or at predetermined control points, to outputan intermediate result and be able to continue workingusing intermediate and incomplete results. Anytime al-gorithms are increasingly used in a number of practicalareas including: planning, deep confidence networks,evaluation of impact diagrams, processing queries todatabases, monitoring and collecting information, etc.This approach can make of decisive importance forcomplex IDSS RT, with a large number of sensorscapable of analysis, and large numerical complexity ofthe scheduling algorithms to obtain optimal solutions ina limited time and can significantly improve the system’sproductivity and efficiency [10].
III. IMPLEMENTATION OF INTELLIGENTFORECASTING SUBSYSTEM OF REAL-TIME
A. Sub-module of prediction
On the basis of statistical and expert methods of fore-casting was suggested combined (integrated) predictionmethod [11], which contains of an averaging the resultsobtained on the basis of the moving average method andthe Bayesian approach, based on weighting coefficients.Then, resulting prediction corrected by values of seriesobtained by the method of exponential smoothing. Afterthat, forecast adjusted by results of the expert methods:ranking and direct evaluations. The probability of eachoutcome acquired by statistical methods, increased ordecreased depending on the expert assessment values forthese outcomes.
The forecasting sub-module is based on the methodsand algorithms described above.
Figure 1. Architecture of sub-module of prediction
The sub-module has two paths: the main and thealternative. Under normal system conditions, sequentialresults obtained by each algorithm are collected andcalculated. After that, the analysis and the formation ofthe final result of the calculation is performed.
Throughout the entire calculation process, the statusof the system, the available resources and the presenceof time constraints are monitored, during which it is nec-essary to form the result immediately, using the anytimealgorithm. In such situations, the process proceeds alongan alternative path when the calculations are transferredto the background (if possible) or stopped and a quick
202
result is generated based on the performed calculationsby parts of the system at the current time.
As a result, each of the ways forms the final result:"full" in the case of normal operation of the system and"approximate" in the presence of instabilities. Finally,the results are stored in a database for use in the nextiterations of the subsystem.
B. Sub-module of deep reinforcement learning
Reinforcement learning sub-module consist of thegroup of independent agents that learning on the basisof a developed TD-methods (TD (0), TD (λ), SARSA,Q-learning).
Sub-module represents of a multi-agent network, thatlearning in parallel by various algorithms are divided intotwo networks also learning in parallel - one determinesthe behavior and second the objective function. Eachagent consist of several additional intermediate hiddenlearning layers created between the input and outputlayer. Also each agent storing separate data in experi-ence replay memory where the data can be batched orrandomly sampled from different time-steps [12, 13]. Thesub-module also has two paths: the main and the alterna-tive. Under normal system conditions, agents are learningin parallel. After the end of episode data is collecting andanalyzing, the gradient descent is calculating. Networkbecome completely updated, formation of the final resultof the calculation is performed.
Under conditions of severe time or resources con-straints, system switches to alternative path: the mile-stone method is apply to the system. In this method algo-rithm chooses which of the paths is the most promising,relative to the accuracy of the forecast and the executiontime, and calculates the result only by methods capableof obtaining the necessary optimal results at the currentmoment. In this case, all other steps can also be executedin the background and could be included in the analysisin the next steps As a result, each of the ways formsthe final result: "full" in the case of normal operationof the system with completely updated network and"approximate" in the presence of instabilities. Finally,the results are stored in a database for use in the nextepisodes of the subsystem.
C. Architecture of prediction subsystem
Proposed architecture (Fig. 3) of prediction subsystemincludes:
• emulator, which simulates the state of the envi-ronment with using of various system parameterschange algorithms (linear and random) in the on-line database. Emulator capable simulate differentconstraints for the system such as time and resourceconstraints;
• prediction sub-module based on statistical methods(extrapolation method of moving average, expo-nential smoothing and the Bayesian approach) and
Figure 2. Architecture of sub-module of deep reinforcement learning
forecasting expert methods (ranking and direct eval-uation). Sub-module also contains monitoring sub-module, that capable of generating fast results andanalyzing sub-module;
• multi-agent RL-learning module consist of thegroup of independent agents that learning on thebasis of a developed TD-methods (TD (0), TD(λ), SARSA, Q-learning) divided into two networkslearning in parallel. Sub-module also contains mon-itoring sub-module, that of generating fast resultsand analyzing sub-module;
• decision-making module designed for the data anal-ysis coming from the prediction module and multi-agent RL-learning module, making decisions onfollow-up actions and adjusting management strate-gies;
• module that collecting and analyzing statistic forvaluation the effectiveness and performance of thesystem;
• monitoring sub-module based on milestone anytimealgorithm, that analyze system state and could ini-tialize getting fast result from all sub-modules;
IV. CONCLUSION
In the paper, the basic idea and main methods for anytimealgorithms, capable of finding a compromise between thecomputation time and the quality of the solution, that allowscalculating approximate solutions of complex problems undertime constraints and their basic methods in terms of use in ISRT (the systems of type IDSS RT) were described [14].
Architecture of intelligent forecasting system of real-time[15], consist of sub-module of prediction, sub-module of rein-forcement learning and main decision-making and monitoringmodule were proposed.
The sub-module of prediction contains of statistical, expertmethods and monitoring module.
203
Figure 3. Architecture of forecasting subsystem
The multi-agent RL-module, contains of a group of inde-pendent agents, each of that is learning in parallel on the basisof one of the developed TD-methods (TD(0), TD(λ), SARSA,Q-learning) as well as used for the accumulation of knowledgeabout the environment and capable of adaptation, modificationand accumulation of knowledge. Also each agent has hiddenlayers that created between the input and output layer, separateagent’s storing data in an experience replay memory and afterthe end of the episode, the gradient descent is calculated.
The decision-making module is designed to analyze the datacoming from the forecasting and RL modules, making deci-sions on follow-up actions and methods to adjust managementstrategies.
Monitoring module based on milestone anytime algorithm,could obtain approximate fast results in the situations with timeand resource constraints.
The approach based on the integration of learning, decision-making and monitoring modules was applied in the develop-ment of a emulator prototype of IDSS RT for monitoring andcontrol of one of the subsystems of the nuclear power plant unit.The considered tools can be used to implement the possibilityof self-learning and adaptation both in the intelligent systemsof real-time created on their basis, and in the actual tools forcreating such systems. It is planned to include a module thattakes into account temporal reasoning formalization and use theOSTIS-technology for development of the intelligent systemsbased on knowledge is given [16-18]. The performed studiesare based on the results of previous studies of the authors,including the implementation of an earlier joint project on "Theformalization of temporal reasoning in intelligent systems",supported by RFBR and BRFBR [16-17].
REFERENCES
[1] R.S. Sutton, A.G.Barto. Reinforcement Learning. – London, The MIT Press,2012, - 320 p. (Russ. Ed. Moscow: BINOM., 2011).
[2] Vagin V.N., Yeremeyev A.P. Some Basic Principles of Design of IntelligentSystems for Supporting Real-Time Decision Making // Journal of Computerand Systems Sciences International. - 2001. - No 6. Pp. 953-961.
[3] Osipov G.S. Methods of artificial intelligence. – 2nd edition. – Moscow.:FIZMATLIT, 2015 (in Russian).
[4] L. Busoniu, R. Babuska, and B. De Schutter, «Multi-agent reinforcementlearning: An overview» Chapter 7 in Innovations in Multi-Agent Systemsand Applications – 1 (D. Srinivasan and L.C. Jain, eds.), vol. 310 ofStudies in Computational Intelligence, Berlin, Germany: Springer, 2010. -Pp. 183–221.
[5] Mnih V., Badia A.P., Mirza M., Graves A., Harley T., et al.: AsynchronousMethods for Deep Reinforcement Learning. In: Proceedings of The 33rdInternational Conference on Machine Learning (PMLR 48), 1928-1937(2016).
[6] Nikolenko S., Kadurin A., Archangelskaya E.: Deep Learning. Immersionin the world of neural networks (in Russian). St. Petersburg: PITER (2017).
[7] Guo H. Generating Text with Deep Reinforcement Learning, arXiv, 2015.http:// arxiv.org/abs/1510.09292.
[8] Eremeev A.P., Kozhukhov A.A.: About implementation of machine learningtools in real-time intelligent systems (in Russian). Journal of Software andSystems, 2, 239-245 (2018).
[9] Hansen E. A., Zilberstein S.: Monitoring and control of anytime algorithms:A dynamic pro-gramming approach. Journal of Artificial Intelligence, 126,139–157 (2001).
[10] Mangharam R., Saba A.: Anytime Algorithms for GPU Architectures. In:IEEE Real-Time Systems Symposium (2011).
[11] Sort J., Singh S., Lewis R.L.: Variance-based rewards for approximateBayesian reinforce-ment learning. In: Proceedings of Uncertainty in Artifi-cial Intelligence, 564-571 (2010).
[12] Mnih V., Kavukcuoglu K., Silver D., Rusu A.A., Veness J., BellemareM.G., et al.: Human-level control through deep reinforcement learning.Nature 518, 529–533 (2015).
[13] Li Y.: Deep Reinforcement Learning: An Overview, arXiv, 2017.http://arxiv.org/abs/ 1701.07274.
[14] Likhachev M., Ferguson D., Gordon G., Stentz A., Thrun S.: AnytimeDynamic A*: An Anytime, Replanning Algorithm. In: Proceedings of theInternational Conference on Auto-mated Planning and Scheduling (ICAPS),262-271 (2005).
[15] Eremeev A.P., Kozhukhov A.A.: About the integration of learning anddecision-making models in intelligent systems of real-time. In: Pro-ceedingsof the Second International Scientific Conference “Intelligent InformationTechnol-ogies for Industry” (IITI’18). Springer. Volume 2. Pp. 181-189(2018).
[16] Vagin V.N., Eremeev A.P., Guliakina N.A.: About the temporal reasoningformalization in the intelligent systems: In: Proceedings of the InternationalConference on Open Semantic Technologies for Intelligent Systems (OSTIS-2016). Minsk. Pp. 275-282 (2016).
[17] Golenkov V.V., Gulyakina N.A., Grakova N.V., Nikulenka V.Y., EremeevA. P., Tarasov V.B.: From training intelligent systems to training theirdevelopment means: In: Proceedings of the International Conference onOpen Semantic Technologies for Intelligent Systems (OSTIS-2018). Minsk.Vol. 2, N 8. Pp. 81-99 (2018).
[18] Eremeev A.P., Kozhukhov A.A., Golenkov V.V., Guliakina N.A.: On theimplementation of the machine learning tools in intelligent systems of real-time // Journal of Software and Systems. - 2018. – Vol. 31, No. 2. Pp. 81-99(in Russian).
РЕАЛИЗАЦИЯ ИНТЕЛЛЕКТУАЛЬНОЙПОДСИСТЕМЫ ПРОГНОЗИРОВАНИЯ РЕАЛЬНОГО
ВРЕМЕНИ
А. Еремеев, А. Кожухов, Н. Гулякина
В статье описывается архитектура интеллектуальной под-системы прогнозирования реального времени, основаннаяна мультиагентном обучении с подкреплением на основевременных различий, статистическом модуле и модуле мо-ниторинга на основе гибких алгоритмов. Анализ гибкихалгоритмов проводился с точки зрения использования вподсистеме прогнозирования типа интеллектуальных системподдержки принятия решений реального времени, для повы-шения производительности и уменьшения времени откли-ка и выполнения. Предложенные инструменты могут бытьиспользованы для реализации возможности самообучения иадаптации как в интеллектуальных системах реального вре-мени, созданных на их основе, так и в реальных инструментахдля создания таких систем. Работа выполнена при поддержкепроектов РФФИ 17-07-00553 а, 18-51-00007 Бел-а.
Received 24.12.18
204
Multi-criteria evaluation of managementdecisions in the intellectual system of
transportation managementAleksandr Erofeev
Belarusian State University of TransportGomel, Belarus
Abstract—In the article is proposed to use a multi-criteriaquality assessment with dynamically changing significancecoefficients while searching for a rational managementdecision in the control system of transportation process. It isproposed to compare the solutions using the theory of fuzzysets. The fuzzy-set member function is built using directand indirect methods. The search conditions for rationalmanagement decisions are formulated.
Keywords—railway transport, transportation process,multicriteria optimization, fuzzy set theory, rational deci-sion search model
Traditionally, in transport, the evolution of controlsystems is realized through informatization and au-tomation. However, information systems and automatedcontrol systems during performance not only collect,but also simplify the initial data. Their activities areaimed at preparing information in accordance with apredetermined template and the subsequent presentationof aggregated information to a person for making man-agement decisions (MD). Such approaches are effectivewhen a limited list of typical tasks is solved accordingto predetermined criteria. However, in conditions whenthere is a need to solve non-trivial tasks, with indefiniteoptimality criteria, solving problems in conditions of lackof time for making decisions and huge amount of data,the effectiveness of traditional systems is significantlyreduced and it becomes necessary to use intelligent trafficcontrol systems (ITCS) [1], [2].
ITCS is recomended for overcoming information bar-riers and for tasks that cannot be solved with the help ofordinary management tools.
The following groups of tasks can be solved usingITCS:• the solution algorithm is unknown and it is neses-
sary to create a new problem solver on the basis ofthe available data;
• besides digital data, it is necessary to use non-formalized or poorly formalized source data (forexample, bad weather conditions during cargo op-erations, low qualification of the locomotive crew,etc.);
• problem solving requires using of an unconventionalmathematical apparatus (cognitive logic, soft com-
puting, etc.);• it is necessary to find a management solution (or
options for management decisions) with uncertainty,incomplete or insufficiently reliable source data (forexample, developing a daily cargo handling planwith an incomplete array of applications for loadingand an unknown category of wagon availability);
• when a criterion for the effectiveness of a manage-ment decision is a new criterion or is a group ofcriteria that was not used in the original algorithms.
Management quality assesment will differ in the ITCSfrom ones in traditional control systems. Besides assess-ing of the management effectiveness (result assessment),management actions, the resources of implementation(actions assessment) and the effectiveness of the com-position, structure, and number of elements in the man-agement system (assessment of the structure) should beevaluated in the ITCS.
When solving individual problems of transportationprocess management, it is not always possible to figureout single optimality criterion. More often it is necessaryto operate with complex criteria with different weightcoefficients.
“Fig. 1”presents a diagram of the customer require-ments priorities for the transportation system.
Figure 1. The priority diagram of customer requirements to thetransportation system: 1 - the cost of the organization of transportation;2 - ensuring the required time of delivery of the goods; 3 - safety ofrolling stock and cargo; 5 - information support of transportation; 5 -compliance with the contractual terms; 6 - flexibility settlement system.
Since different units of measurement and different205
methods for their determination are used for differentevaluation criteria, a technique that allows them to beintegrated is proposed. Table 1 presents the criteria forassessing the transportation organization system quality,as well as methods and information sources for theirdetermination.
Table IQUALITY CRITERIA FOR THE TRANSPORTATION SYSTEM
Criteria Calculation meth-ods
Source of informa-tion
The cost of the or-ganization of trans-portation
Calculationmethods, contract
Operational param-eters of the railway,cost meters
Ensuring therequired time ofdelivery of thegoods
Calculationmethods
Delivery route,speed standards,shelf life, etc
Availability of re-serves of infrastruc-ture facilities
Calculationmethods
Operational param-eters of the railway
Compliance withthe contractualterms betweenthe infrastructureoperator andcarriers
Registration meth-ods
Statistical data
Note: Depending on the situation, the number and the listof criteria may vary.
As can be seen from table 1, various methods canbe used for evaluation, which are applicable to clearlydefined quantitative estimates of quality parameters.However, in some cases a subjective assessment can beapplied - “desirable”, “within”, etc.
In the scientific literature, including the transport one[3], [4], the mathematical apparatus based on the theoryof fuzzy sets [5], [6] is used as a tool for expressingunclearly defined customer expectations.
The approach to the formalization of fuzziness isas follows. A fuzzy set is formed by introducing ageneralized concept of belonging, i.e. extensions of thetwo-element set of values of the characteristic function0,1 to the continuum [0,1]. This means that the transitionfrom the complete belonging of an object to a class toits complete non-belonging does not occur in jumps, butsmoothly, gradually, and the belonging of an element toa set is expressed by a number from the interval from 0to 1.
Consider the use of the specified mathematical appa-ratus for the evaluation of MD in ITCS.
Let X – be a set of variants of MD according to somecriterion of quality in the ITCS. The fuzzy set A in Xis the set of pairs of the form (x, µA(x)), where x ∈X , and µA(x) - is the level of achievement of a givenfuzzy target by the variant X. µA(x) - the membershipfunction of a fuzzy set A, varying from 0 to 1. The greaterthe value of the membership function, the greater the
degree of achievement of a given goal when choosingan alternative X as a solution.
The membership function is set on the basis of expertassessments and can have a different look. For theconditions of the problem of choosing a rational MD inthe ITCS, formalization of fuzzy consumer expectationsin accordance with [7], [8] we will consider the normalfuzzy set, i.e. the upper limit of its membership functionis equal to one: supµA(x) = 1, and the membershipfunction itself is inseparable.
Depending on the quality parameter under considera-tion and consumer preferences, the membership functionmay have a certain interval, where µA(x) = 1. If thevalue of the quality parameter is subject to “no more” or“no less” restrictions, the membership function assumesa zero value when this condition is not met. For example,when setting infrastructure constraints xi <= c, then inthis case µA(xi <= c) = 0.
There are a number of methods for constructing,according to expert estimates, the membership functionsof a fuzzy set, which can be divided into two groups:direct and indirect.
Direct methods are determined by the fact that theexpert sets the rules for determining the values of themembership function µA(x) characterizing the conceptA. These values are consistent with his preferences onthe set of objects U as follows:• for any u1, u2 ∈ U, µA(u1) < µA(u2) if and only
if it is u2 preferable u1, i.e. more characterized bythe concept A;
• for any u1, u2 ∈ U, µA(u1) < µA(u2) if and only ifand are indifferent u2 with u1 respect to the conceptA.
Examples of direct methods are the direct assignmentof the membership function by a table, a formula, asample.
In indirect methods, the values of the membershipfunction are chosen in such a way as to satisfy thepreviously formulated conditions. Expert information isonly the initial information for further processing. Inthe ITCS, in the process of monitoring, the values offunctions dynamically change.
Additional conditions may be imposed both on thetype of information received and on the processingprocedure. Examples of additional conditions are thefollowing: the membership function should reflect theproximity to a pre-allocated standard; objects of theset U are points in parametric space; the result of theprocessing procedure should be the membership functionthat satisfies the conditions of the interval scale, etc.
As a rule, direct methods are used to describe conceptsthat are characterized by measurable properties. In thiscase, it is convenient to directly specify the values ofthe degree of belonging. The procedure for constructingthe membership function consists of the following steps:
206
determining the type of function; establishing its specificvalues; adequacy check.
When determining fuzzy expectations, the experts aregiven some values of the quality parameter and thecorresponding values of the membership function, whilethe remaining intermediate values are determined by theinterpolation method.
Multi-criteria optimization in a fuzzy setting can berepresented as a system 〈X,C1, ..., Cn, L〉, where X – isa universal set of alternatives, L –is a lattice, and (i =1, n the criterion is called L -fuzzy set
µCiinFL(X), FL(X) = µCi
|µCi: X → L, (1)
where is µCi– the membership function, fuzzy set;
FL(X) – many fuzzy subsets X.If all criteria are considered equivalent and compara-
ble, then, in accordance with the principle of merging,we have 〈X,D,L〉, where is D = C1 ∩ ... ∩ Cn, i.e.µD = µC1
· µC2· ... · µCn
( · - one of the variants of theoperation of intersection of fuzzy sets in FL(X) ).
However, in real conditions, including when choos-ing a rational MD, criteria of unequal significance areused. Then, if there is a set of fuzzy criteria M =µC1 , ..., µCn, µCi ∈ FL(X) and a set of weights ofcriteria Π = P1, ..., Pn, then a fuzzy subset Q of thefuzzy set M : Q ⊂M
µQ(µCi(x)) =
Pi, если Ci ∈M,0, если Ci /∈M (2)
determines the weighting criteria.The criteria weighting procedure is considered as a
mapping v : P (Nn) → L, where is P (Nn) the set ofall subsets of criteria indices Nn = 1, ..., n, L, – is agrid.
The function D : X → L that represents the solutionsis determined using a fuzzy integral.
D =
∫
Nn
v · g(·) = supM∈P (Nn)
infi∈M
(vx(i) ∧ g(M)). (3)
In the multicriteria case, the objective function is avector function φ(x) = (φ1(x), ..., φm(x)), i.e. φ : X ⊂Rn → Rm where is R – the set of real numbers, andthe strict order Rm is impossible. Any two alternativesx and y are comparable with each other if and only ifphii(x) ≥ φi(y) , either, or phii(x) ≥ φi(y)∀i. Thus, theconcept of optimality is replaced in vector optimizationby the concept of non-dominance. While in a single-criterion problem, the solution is an optimum point, ina multicriteria problem it gives a lot of effective (Paretooptimal) alternatives
P 0 = x0 ∈ X|∀y ∈ X,φi(y) ≥ φi(x0 → φi(y) =
= φi(x0); i = 1, n). (4)
In order to further narrow this set, additional infor-mation from the ITCS knowledge base is needed. Thevarious procedures are used in this case basically boildown to explicit or implicit particular criteria folding intoa single criterion.
Examples of such generic criteria include [4]:• weighted sum of fuzzy criteria
C =
n∑
i=1
ωiCi; (5)
• the product of the form
C =
n∏
i=1
ωiCωii ; (6)
• minimum relationship
C = mini=1,...,n
(Ci/ωi); (7)
where Ci - normalized criteria (unclear Bellmantargets); ωi - relative criteria weights, i = 1, n .
A fuzzy formulation of a multi-criteria choice prob-lem implies that a number of compared alternatives areknown.
(Version MD) A = a1, ..., ai, ..., an and manycomparison criteria (quality assessment parameters) C =c1, ..., cj , ..., cm, where between each member of theset A and each member of the set C fuzzy relationshipaicj or µij , which reflects the level of compliance of thedelivery i option with the j parameter.µij ∈ [0, 1]; i = 1, n; j = 1,mAll fuzzy relations between and form a matrix of fuzzy
relations of size nm :
R = µij |i = 1, n; j = 1,m, (8)
and the objective function is
a∗ = (A,C,R,M), (9)
where M – used problem solving model chosen by theITCS.
The search model for a rational MD can be definedby the following conditions:• choice of MD in the absence of information on
restrictions on the value of parameters and informa-tion on the level of their not worse than the required;
• the choice of MD when imposing desirable restric-tions on some importance;
• the choice of the MD, providing the values of allparameters basic parameters;
• the choice of MD in the presence of informationabout the level of parameters importance and theirshare of influence on the overall decision.
The last condition fully characterizes the problem ofchoosing an option of MD. To solve such problems a
207
compromise solution model is used. The essence of thismodel is that due to the impossibility of simultaneouslysatisfying several partial quality criteria, the decision ismade using an integral (compromise) indicator obtainedby folding particular parameters using formulas ( 5 ) -( 7 ). Then problem ( 9 ) is transformed into the followingform:
a∗ = ai|ai ∈ A; ci = maxci|ci ∈ C; i = 1, n. (10)
The algorithm for solving this problem is:• to establish the level of importance of parametersωi, i = 1, n , ( takes a value from 0 - the parameterhas no effect on the choice of the delivery systemto 1 - the parameter has the maximum influence onthe choice of the delivery system);
• to normalize of values ωi , i.e. to calculate ωi =ωi/
∑k = 1n;
• to calculate the value of an integral parameter foreach variant, for example, from expression ( 5 ) ;
• determining the maximum value of integral param-eter.
In real conditions it is not always possible to figureout an exhaustive group of criteria and establish theirlevel of significance. Therefore, when developing a ITCS,it is necessary to operate not only with criteria andspecified parameters, but, first of all, with the rules oftheir formation and change.
When solving the tasks of managing the transporta-tion process, the significance of the evaluation criteriawill vary depending on the prevailing operational envi-ronment. The intellectual function of the ITCS is thedynamic formation of a fuzzy relationship matrix ( 8 ) .n it, variables are not only the values of the coefficientsof importance of the criteria, but also the size of thematrix (n variants of MD, m criteria of comparison).
The procedure for the formation of the matrix includesthe following steps:• monitoring the operational situation and the for-
mation of a matrix of states of objects of thetransportation process (for example, the values ofdeviations of trains from the standard schedule ofmovements);
• the formation of the conditions of the operationaltask on the basis of the knowledge base of the ITCSand the state matrix of the objects of the transporta-tion process (if deviations from the schedule of alltrains are insignificant, then we use the reserves oftrain times;
• of the deviations from the schedule of the majorityof trains are insignificant, and for some significantones, we adjust the train schedule threads by chang-ing the station modes;
• if a significant number of trains have deviationsfrom the standard schedule - we edit the entireschedule);
• selection of criteria for solving the set operationaltask on the basis of the ITCS knowledge base andthe matrix of states of the transportation processobjects (for example, for the development of atimetable, it may be the speed of a train, reliabilityof the schedule, timely arrival of priority trains,energy costs for train movement, the need for lo-comotives and so on).
REFERENCES
[1] Erofeev A.A. Predposyilki sozdaniya intellektualnoy sistemyiupravleniya perevozochnyim protsessom. Vestnik Belorusskogogosudarstvennogo universiteta transporta: Nauka i transport. # 1(34), 2017. S. 42-45.
[2] Erofeev A.A., Erofeeva H. Intelligent management of the rail-way transportation process: object model. Otkryityie semantich-eskie tehnologii proektirovaniya intellektualnyih sistem: materi-alyi mezhdunarodnoy nauchno-tehnicheskoy konferentsii. Red-kol. Golenkov V.V. [i dr.]. Minsk, BSUIR, 2017. S.281-284.
[3] Mirotin L.B. Logistika: upravlenie v gruzovyih transportno-logisticheskih sistemah. M. «Yurist’», 2002. 348 s.
[4] Nechetkie mnozhestva v modelyah upravleniya i iskusstvennogointellekta. Pod red. D.A.Pospelova. M. Nauka, 1986. 312 s.
[5] Zadeh L.A. Fuzzy sets. Information and control, 1985. v.8, p.338.[6] Zadeh L.A. Similariti relations and fuzzy restrictions. Information
Sciences, 1971, v.3, p. 166-200.[7] Nikolashin V.M. Metodologiya organizatsii proizvodstva i funk-
tsionirovaniya transportno-logisticheskih kompleksov. Dis.dokt.teh. nauk. M. MIIT, 2001. 314 s.
[8] Osnovyi logistiki. Pod red L.B.Mirotina i V.I.Sergeeva. M.INFRA-M. 2000, 200 s.
МНОГОКРИТЕРИАЛЬНАЯ ОЦЕНКАУПРАВЛЕНЧЕСКИХ РЕШЕНИЙ ВИНТЕЛЛЕКТУАЛЬНОЙ СИСТЕМЕУПРАВЛЕНИЯ ПЕРЕВОЗОЧНЫМ
ПРОЦЕССОМ
Ерофеев А.А.
Аннотация: установлено, что при поиске рацио-нального управленческого решения в системе управ-ления перевозочным процессом необходимо использо-вать многокритериальную оценку качества с динами-чески изменяющимися коэффициентами значимости.Сравнение вариантов решений предложено произво-дить с использованием математического апппарататеории нечетких множеств. Формирование функциипринадлежности нечеткому множеству предложенопроизводить с использованием прямых и косвенныхметодов. Сформулированы условия поиска рациональ-ных управленческих решений.
Ключевые слова: перевозочный процесс, организа-цияперевозок, многокритериальная оптимизация, тео-рия нечетких множеств, модель поиска рациональногорешения
Received 27.12.18
208
Methods and technologies for assessing theimpact of energy on the geoecology of a region
(using the examples of the Baikal region(Russia) and Belarus)
Massel L.V., Massel A.G.Energy Systems Institute of SB RAS
Irkutsk, [email protected], [email protected]
Zorina T.G.Institute of Energy NAS Belarus
Minsk, [email protected]
Abstract—The article examines the results of an inter-national project carried out with the support of the EAPIFund together with researchers from Belarus and Armenia.The project aims to develop methods and technologies forassessing the impact of energy on the geoecology of theregion. The article is devoted to the development of toolsfor intelligent support of decision-making in this field.
Keywords—intelligent support of decision-making, en-ergy, ecology, quality of life
I. INTRODUCTION
Studies of the impact assessment of energy on thegeoecology of the region [1, 2] are conducted withinthe framework of an international project supported bythe EASR1-RFFI funds, in cooperation with the teams ofscientists of Belarus and Armenia. The article examinesthe main provisions and results of the project carried outby the Russian side2.
The fundamental scientific problem addressed by theProject is the development of methods and geoinforma-tion technologies for assessing the impact of energy onthe geoecology of the region [3]. The object of researchfrom the Russian side is the Baikal natural territory,comparable in size to Belarus and Armenia.
The article is devoted mainly to the developmentand integration of modern information technologies forintelligent decision-making support within the frame-work of the problem. To implement the project, a Web-based information system (WIS) is being developed thatintegrates mathematical and semantic methods [4, 5]and tools for assessing the impact of energy on thegeoecology of the region, a database, a knowledge base
1EASR – The Eurasian Association for the Support of ScientificResearch, established in July 2016 on the initiative of the RussianFoundation for Basic Research in cooperation with partner organiza-tions of Belarus, Armenia, Kyrgyzstan and Mongolia.
2The international project "Methods and technologies for the impactassesment of energy on the geoecology of the region" is being carriedout, on the Russian side, with the support of the RFBR grant No 18-57-81001, under the leadership of L.V. Massel.
and a geographic information system. Individual WIScomponents will be implemented as agents- services [6].The ontology of the energy impact on the environment,ontology of pollutants and WIS architecture is presentedin the article.
The aim of the project is to develop methods andtechnologies for assessing the impact of energy on thegeoecology of the region. Geoecology is understood asan interdisciplinary scientific field that unites researchinto the composition, structure, properties, processes,physical and geochemical fields of Earth’s geospheresas a habitat for humans and other organisms. The maintask of geoecology is to study the changes in the life-supporting resources of the geosphere shells under theinfluence of natural and anthropogenic factors, theirprotection, rational use and control in order to preserveproductive and natural environment for present and futuregenerations [1, 2].
The purpose of this project fits into the main task ofgeoecology. The relevance of the project is determined,on the one hand, by the importance of the problem ofassessing the impact of energy on the geoecology of theregion, and on the other, its insufficient research andthe need to attract modern geoinformation and intelligenttechnologies for its solution.
In the course of the project, the formulation of themain specific tasks to be solved in the project wasclarified and expanded: 1) to conduct an analysis ofexisting methods for estimating of pollutant emissionsfrom energy facilities and existing models for the spreadof pollution caused by emissions from energy objects(taking into account the wind rose, transfer, etc.); 2) tomake a choice and justify the methods recommended foruse in the project, their modification and adaptation, thedevelopment, if necessary, of the original methods; 3)to identify critical objects that affect the life supportand natural environment of the region (in the energy,
209
water supply, etc. sectors), the connection of criticalfacilities with the quality of life of the population [7];4) to analyze the approaches to the construction ofgeoinformation systems, design a geoinformation systembased on 3D geo-visualization, determine the types ofinterfaces for displaying and analyzing information; 5) todetermine the composition of information necessary forthe use of recommended methods, to identify sources ofinformation, assess their availability and financial costsof acquiring information; collection and structuring ofnecessary information; design and implement a database;6) to develop the architecture of a Web-based informationsystem (WIS) that integrates mathematical and semanticmethods and tools for assessing the impact of energy onthe geoecology of the region; database, knowledge baseand geoinformation system; to develop a knowledge basestructure within WIS; 7) to develop a system of ontolo-gies [8] for describing the domain, to adapt and developtools for semantic modeling, to construct semantic mod-els for assessing the impact of energy on the geoecologyof the region; 8) carry out approbation of WIS and applythe developed methods and technologies to decisionmaking support on the justification and development ofrecommendations for the development of energy takinginto account the requirements of geoecology.
II. THE IMPACT OF ENERGY ON THE ENVIRONMENT
In recent years, the problem of the impact of energyon the environment has become widespread in the sci-entific world. Various scientists are trying to investigatethe negative consequences of the functioning of energyenterprises for geoecology and identify areas of harmfulinfluence. Below the results of Russian and foreignscientists are considered.
According to Vorobyov V.I. (Russia), the analysis ofexisting principles for the design and development oflarge thermal power plants (TPPs), as well as opti-mization models described by different authors, showsthat they do not take into account the actual effects ofair pollution, since the specific placement in the thearea of settlements (especially in built-up areas) thatfall into the contaminated zone. On the base of a full-scale instrumental survey of the urban area, VorobyovV.I. determined the concentrations of harmful impuritiesat various distances from the pollution source – theterritory of the TPPs, the sanitary protection zone, theresidential development, confirming the exceeding of themaximum permissible concentration at a distance of upto 18 km. Arslanbekova F.F. (Russia), who investigatedthe damaging effect on the environment of thermal powerplants (TPPs) and motor vehicles, believes that the zoneof the most intense air pollution by harmful impuritiesunder torches of TPPs reaches a radius of 3-5 km.Nikiyenko Yu.V. (Russia) investigated the main pointsof the influence of thermal pollution on the microclimate
of the adjacent territories. Based on the calculations, sheconcludes that the presence of a cooling pond in the areawhere NPPs and TPPs are located will inevitably leadto negative environmental consequences, including max-imum temperature, precipitation and relative importanceanomalies. Kozhanov A.A. (Russia) offers methods ofgeoecological assessment of the influence of the fuel andenergy system, based on establishing the relationship be-tween natural conditions and antropogen impact. Studieson the interconnection of Energy, Environment, ClimateChange are also conducted abroad, for example, [9-12],the use of GIS-technologies and 3D-geovisualization areconsidered in [13, 14].
Nevertheless, in studies related to the assessment ofthe impact of energy on the environment, the quality oflife has not been considered so far.
III. THE QUALITY OF LIFE
This concept was first introduced into the scientificrevolution in the 60s of the last century in connectionwith attempts by foreign researchers to model the trajec-tories of industrial development. There are many differentdefinitions of the quality of life, but this concept is mostfully disclosed in the context of health care. Quality oflife is understood as a set of objective and subjectiveparameters that characterize the maximum number ofsides of a person’s life, his position in society and hissatisfaction with him. Among the factors determining thequality of life according to the definition of the WorldHealth Organization [15], not only financial well-being,but the state of security, health, human position in soci-ety, ecology and, most importantly, its own assessment ofall these factors is included (Fig. 1) The integral indicatorof the quality of life summarizes the indicators of health,social-personal well-being and financial well-being. Inthe framework of our project, it is important that ecologyis included in the quality of life indicators.
Figure 1. Quality of life as defined by World Health Organization
Under the guidance of the author, work has been donein which it is justified that it is difficult to obtain anintegral index using rigorous mathematical methods. Itis suggested to involve artificial intelligence methodsfor this purpose, namely, cognitive modeling is one ofthe directions of semantic modeling [5, 16]. In addition,
210
quality of life research in the integrated quality oflife indicator has not taken into account the impact ofenergy supply until recently, while a shortage of energyresources can have a significant impact on both the leveland quality of life. It is suggested to include externalfactors in the quality of life indicators, in particular, thedegree of provision with energy resources (Fig. 2) [17].
Figure 2. Cognitive map of indicators of life quality using procedureSF-36 (all links are positive)
Comments on Fig. 2:• PF (Physical Functioning) – physical functioning;• RP (Role-Physical Functioning) – role functioning
conditioned by physical state;• BP (Bodily pain) – pain intensity;• GH (General Health) – general health;• VT (Vitality) – life activity;• SF (Social Functioning) – social functioning;• RE (Role-Emotional) – role functioning precondi-
tioned by emotional state;• MH (Mental health) – Mental health;• PHC (Physical health) – general component of
physical health;• MHC (Mental health) – general component of men-
tal health;• QoL (Quality of Life) – integral indicator of quality
of life;• COEr (DSEr) – degree of supply with energy re-
sources.
IV. PROPOSED METHODS AND APPROACHES TOSOLVING THE PROBLEM
In the development of tools for intelligent support ofenergy and environmental decision-making, the proposedproject is based on the application of methods of geoin-formation technologies based on 3D geo-visualization[18], critical infrastructure research methods [7], decisionsupport methods, knowledge engineering methods, ob-ject approach methods (analysis, design, programming),system and application programming methods, designmethods of database ,of information and expert systems,as well as author methods of situational management,
semantic modeling (primarily ontological and cognitive)and intelligent technologies for desicion-making support[5]. It’s proposed to develope and adaptate to the projecttheme the author’s methods of constructing an ontolog-ical space of knowledge in the field of energy; methodsof semantic (ontological and cognitive) modeling inpower engineering, methods of 3D-geovisualization andmethods of visual analytics with elements of cognitivegraphics, as well as methods of intelligent systems devel-opment for supporting the adoption of strategic decisionsin energy [19].
As an illustration of the approach to constructing aontologies system of applied domain, Fig. 3-4 presentthe ontology of the impact of energy on the environment(Fig. 3) and the ontology of pollutants from energyfacilities (Fig. 4)
From Fig. 4 it can be seen that energy companies(enterprises of electric power industry, heat power en-gineering and fuel and enterprises for energy resourcesextraction and transportation) can pollute water, air andsoil (first of all, the chain associated with air pollutionis considered in the project, see Fig. 3) Negative impacton the quality of life of a person can be either direct orindirect (through the plant and animal world, i.e. throughthe food chain).
In Fig.. 4 shows the first version of the ontology,of pollutants from energy facilities, built on the basisof [20], which rather illustrates the way of structuringknowledge about the subject area. In the following, theontology can be extended or used as a hybrid: some ofits concepts can be considered as meta-concepts, whichwill be described by detailed ontologies: for example,the concept of "Purification" can be represented by de-tailed ontology of the methods and levels of purification;the concept "Combustion method" can be representedby extended ontologies describing different combustionmethods for small and large boiler plants and thermalpower plants, etc.
As mentioned in the introduction, a Web-orientedinformation system (WIS) is being developed to imple-ment the project, integrating mathematical and semanticmethods, tools for impact assessment of energy on geoe-cology in the region, a database, a knowledge base anda geographic information system. It is supposed to usethe authors’ results done earlier to study energy secu-rity problems: semantic modeling tools, Geocomponent,knowledge base tools and the Repository. Individual WIScomponents (Geocomponent, semantic modeling supporttools, individual computational modules, database accesscomponents), can be implemented as agents-services [6].
The WIS architecture is shown in Fig.. 5. There arefour levels in the architecture: 1) the level of mathe-matical methods, models and software, which includesdeveloped on the basis of selected methods and modelsthe software for calculating the volumes of pollutants
211
Figure 3. Ontology of the energy sector impact on the environment
Figure 4. Ontology of pollutants from energy facilities
and their impact on the quality of life of the population,taking into account the capacity of energy facilities (en-ergy supply) and population density in the territory underconsideration; 2) the level of semantic modeling, includ-ing semantic (primarily cognitive) models for describingthe interrelationship of factors that determine the qualityof life, taking into account anthropic-technical factors:the provision by energy resources and the influence ofpollutants from energy facilities on the environment;3) the level of knowledge representation - unites theknowledge base storing descriptions of knowledge forconstructing semantic models, and an ontology systemfor describing knowledge of the subject domain; the lattercan be used both for building a knowledge base andfor database designing; 4) level of data representation- integrates the geographic information system (GIS)and database, including geographic coordinates of energyfacilities. GIS can be used both to illustrate the resultsof calculations, and for visual interpretation of semanticmodels.
The meta descriptions of information presented at allfour levels are stored in the Repository (its scientific
Figure 5. The architecture of the Web-based information system (WIS)to assess the impact of energy on the geoecology of the region
prototype and tools for working with it were developedby Kopaygorodsky A.N., the co-executor of the project). When implementing the user interface, it is intendedto apply the components of the situational managementlanguage CML [21].
The methods listed above are used to assess the impactof energy on the regional geo-ecology (using examplesfrom the Baikal region (Russia) and Belarus). In fig. 6presents the two main territories of the Baikal regionand the Republic of Belarus, which are currently beingstudied.
V. RESULTS AND DISCUSSION
The article is devoted to the actual problem of as-sessing the impact of energy on the geoecology of theregion. As a rule, emissions from energy facilities arenot considered separately in environmental studies, soit is difficult to separate them from general environ-mental pollution. This makes it difficult to plan andimplement measures to reduce pollution by individualenergy facilities. The article proposes an approach tosolving the problem of monitoring the impact of energysector on the geoecology of the region, based on the
212
a)
b)
Figure 6. Display of the energy infrastructure of the facilities of the RUE “Vitebskenergo" (a) and the central ecological zone of Lake Baikal(b)
integration of mathematical models, GIS technologiesand modern intellectual technologies in the frameworkof Web-oriented information system. The novelty of theproject is also determined by the fact that when assessingthe impact of energy on geoecology for the first time itis proposed to take into account the quality of life ofthe population. Cognitive modeling is seen as a tool forimplementing this idea. To organize information support,it is proposed to use the ontology system as a basis fordesigning databases and knowledge bases. The proposedWeb-based system is considered as a prototype of theintelligent decision-making support system for improvingthe quality of life, taking into account the requirementsof geoecology, including an improved analytical tool forestimating emissions of energy facilities and the spreadof pollution.
CONCLUSION
The article considers the International Project, carriedout under the guidance of the author with the supportof the EASR-RFFI funds. The statement of the prob-lem is formulated (the fundamental scientific task andthe project goal), the urgency and expected results ofthe project are determined, and proposed methods andapproaches to its implementation. The main attention inthe article is given to the information and technologicalpart of the project carried out by the Russian side.The illustrations of proposed approaches are presented:cognitive map of indicators for assessing the qualityof life, taking into account the availability of energyresources, ontology of the energy sector impact on theenvironment and ontology of pollutants from energyfacilities, as well as the developed architecture of theWeb-based information system (WIS), which, togetherwith the technology of its use, will be the final resultof the project. The results were obtained with partial
financial support of RFBR grants No 18-57-81001, No19-07-00351, No 18-07-00714 and BRFBR No X18EA-003.
REFERENCES
[1] Geoecology. Geological Dictionary [in 3 volumes] / Ch. Ed. O.V.Petrov. 3rd ed., Revised. and additional. St. Petersburg: VSEGEI,2010, vol. 1. A-I, P. 244. ISBN 978-5-93761-171-0 (in Russian).
[2] Geoecology. Ecological Encyclopedia: [In 6 volumes]. Ch. Ed.V.I. Danilov-Danilyan. Moscow: OOO "Izdatelstvo Encyclopedia", 2010, V. 2. G-I, P. 22 (in Russian).
[3] Massel L.V. The problem of the impact assesment of energy onthe geoecology of the region: setting and solutions. Informationand mathematical technologies in science and management, 2018,No 2 (10). Pp. 5-21. DOI: 10.25729/2413-0133-2018-2-01 (inRussian).
[4] Khoroshevsky V.F. Semantic Technologies: Expectations andTrends. Proceedings of the II International Scientific and Techni-cal Conference "Open Semantic Technologies for the Design ofIntelligent Systems", Belarus, Minsk: BSUIR, 2012, Pp. 143-158(in Russian).
[5] Massel L.V., Massel A.G. Semantic technologies based on theintegration of ontological, cognitive and event modeling. Otkrytyesemanticheskie tekhnologii proektirovaniya intellektual’nykh sys-tem [Open semantic technologies for intelligent systems], 2013,Pp. 247-250 (in Russian).
[6] Massel L.V., Galperov V.I.. Development of multi-agent systemsfor the distributed solution of energy problems using agentscenarios. Izvestiya of Tomsk Polytechnic University, vol. 326,No 5, 2015, Pp. 45-53 (in Russian).
[7] Massel LV Convergence of research on critical infrastruc-tures, quality of life and safety. Information Technologiesand Systems: Proceedings of the 6th International ScientificConference. Chelyabinsk: ChelSU. Scientific electronic publi-cation, 2017, ISBN 978-5-7271-1417-9, Pp. 170-175. URL:http://iit.csu.ru/content/docs/science/itis2017.pdf (circulation dateMay 10, 2017) (in Russian).
[8] Gavrilova TA Ontological engineering. Electronicresource. Technologies of knowledge management. Accessmode: http://www.kmtec.ru/publications/library/authors/ontolog_engeneering.shtml (in Russian).
[9] Energy, Climate Change, Environment/ International EnergyAgency. -2016. - 133 p.
[10] K. Pavlickova , A. Miklosovicova, M. Vyskupova . Effects ofSustainable Energy Facilities on Landscape: A Case Study ofSlovakia // Designing Low Carbon Societies in Landscapes,Ecological Research Monographs, Chapter 7. Eds. N. Nakagoshi
213
and J.A. Mabuhay, © Springer Japan, 2014, Pp. 109-127. DOI10.1007/978-4-431-54819-5_7
[11] Hussey, K., and J. Pittock. The Energy–Water Nexus: Man-aging the Links between Energy and Water for a Sus-tainable Future. Ecology and Society 17(1): 31. 2012.http://dx.doi.org/10.5751/ES-04641-170131
[12] Pfeiffer A., Millar R., Hepburn C., Beinhocker E., 2016. "AppliedEnergy”, Elsevier, vol. 179(C). Pp. 1395-1408.
[13] Alaa Alhamwi, Wided Medjroubi, Thomas Vogt and CarstenAgert. GIS-Based Urban Energy Systems Models and Tools:Introducing a Model for the Optimisation of FlexibilisationTechnologies in Urban Areas/ Applied Energy.191(1 April 2017).DOI: 10.1016/j.apenergy.2017.01.048
[14] Liang, J., Gong, J., Li, W., Ibrahim, A. N., 2014. A visualization-oriented 3d method for efficient computation of urban solarradiation based on 3d–2d surface mapping. International Journalof Geographical Information Science 28 (4), 780–798.
[15] WHO Health Impact Assessment Toolkit - Guidance on healthimpact assessment. World Health Organization, 2005
[16] Trakhtengerts E.A. Computer support for decision-making.Moscow: SINTEG, 1998, 376 p. (in Russian).
[17] Massel L.V., Blokhin A.A. Method of cognitive modeling ofquality of life indicators taking into account external factors.Science and education. Scientific publication of the MSTU.Bauman, No 4, 2016, Pp. 65-754. DOI: 10.7463 / 0416.0839061(13) (in Russian).
[18] Ivanova I.Yu., Ivanov R.A. The use of geovisualization in theanalysis of the location of energy infrastructure objects of thecentral ecological zone of the Baikal natural territory. Informationand Mathematical Technologies in Science and Management.Science Magazine, No 4/2, 2016, Pp. 80-89 (in Russian).
[19] Massel L.V. Problems of the creation of semiotic-type intelligentsystems for strategic situational management in critical infras-tructures. Information and Mathematical Technologies in Scienceand Management. Science Magazine, No 1, 2016, Pp. 7-27 (inRussian).
[20] Maysyuk E.P., Ivanova I.Yu., Ivanov R.A. Assessment of thevolume and composition of emissions of pollutants into theatmosphere from energy objects of the central ecological zone.Proceedings of the International Scientific and Practical Confer-ence "Current state and prospects for improving the ecology andsafety of vital activity of the Baikal region. White Nights, 2016". Irkutsk: Publishing House of the IRSNU, 2016, Pp. 475-482(in Russian).
[21] Massel L.V., Massel A.G. Language of description and knowledgemanagement in the intelligent system of the semiotic type. XXBaikal All-Russian Conference "Information and MathematicalTechnologies in Science and Management": Proceedings, Vol. 3,Irkutsk. MESI SB RAS, 2015, Pp. 112 - 124 (in Russian).
МЕТОДЫ И ТЕХНОЛОГИИ ОЦЕНКИВОЗДЕЙСТВИЯ ЭНЕРГЕТИКИ НА
ГЕОЭКОЛОГИЮ РЕГИОНА (НА ПРИМЕРЕБАЙКАЛЬСКОГО РЕГИОНА (РОССИЯ) И
БЕЛАРУСИ)
Массель Л.В., Массель А.Г., Зорина Т.Г.
В статье рассматриваются результаты международ-ного проекта, осуществленного при поддержке ФондаЕАПИ совместно с исследователями из Беларуси иАр-мении. Проект направлен на разработку методов и тех-нологий оценки воздействия энергии на геоэкологиюрегиона. Статья посвящена разработке инструментовдля интеллектуальной поддержки принятия решений вэтой области.
Ключевые слова: интеллектуальная поддержкапринятия решений, энергетика, экология, качествожизни
Received 11.01.19
214
Design Principles of Integrated InformationServices for Batch Manufacturing
Enterprise EmployeesValery Taberko,
Dzmitry Ivaniuk,Valery Kasyanik
JSC «Savushkin Product»Brest, Republic of Belarus
tab,id,[email protected]
Vladimir GolovkoBrest State Technical University
Brest, Republic of [email protected]
Kirill Rusetski,Daniil Shunkevich,
Natalia GrakovaBelarusian State University
of Informatics and RadioelectronicsMinsk, Republic of Belarusrusetski,[email protected],
Abstract—This paper discusses further applications ofthe ontology-based approach to the design of batch manu-facturing enterprises. It involves, among other things, stan-dards formalization. This paper, in particular, is dedicatedto graphical representation of Piping and InstrumentationDiagram (P&ID) and Procedure Function Chart (PFC)languages, as per ISA-88 standard. They form a toolkit forautomation engineer to work with. Contingency analysisand information retrieval agents were implemented. Thearticle also discusses agent-oriented approach to robotinteraction in robotic production systems, that is conductedvia shared semantic memory.
Keywords—integrated industrial control, information ser-vices, ontology-based enterprise model, Industry 4.0, cyber-physical system, ontology, knowledge base, multi-agentsystem, OSTIS technology.
I. INTRODUCTION
A. Information services
One of the mainline trends of enterprise automa-tion systems development and intellectualization involvesmoving away from isolated systems, each solving theirown problem (CAD, SCADA, MES, ERP, WMS, SCM,CRM, etc.), to more complex integrated systems con-cerned with both enterprise automation and informationsupport for clients and employees alike. Notably, the keytopic of the HANNOVER MESSE 2019 [1] internationalexhibition is industrial intelligence within the context ofintegrated industry.
There are two primary lines of development for suchsystems: PLM (Product Lifecycle Management) systems[2] and CALS (Continuous Acquisition and LifecycleSupport) systems and technologies.
This paper pays particular attention to design prin-ciples of integrated information services for batchmanufacturing enterprise employees through the JSC"Savushkin product" example. This particular approach isbased on using open semantic technologies for intelligent
systems. This paper uses and expands upon the resultspresented in [3] and [4].
Key tasks handled by such systems generally include:• creating and maintaining shared information space
at every stage of product lifecycle;• maintaining information integrity in the shared in-
formation space during product lifecycle;• enabling employees and automated subsystems to
access, control, modify and analyse product infor-mation;
• staff training.
B. Batch manufacturing-related specifics of the informa-tion services
In regards to information service for batch manufac-turing employees, as exemplified by JSC "Savushkinproduct", there are several basic product lifecycle stages:
1) milking;2) shipping milk from a farm to a factory;3) milk processing;4) making final product;5) bottling and packing;6) shipping to a factory warehouse;7) shipping from the factory warehouse to a business
customer’s (store, retail chain, etc.) warehouse;Integrated information services are aimed at the fol-
lowing user classes, as per their production responsibil-ities:
• line operator controls a certain production process;• production shop foreman manages a certain produc-
tion shop;• manufacturing director manages a certain produc-
tion site;• driver delivers raw materials and final products;• production logistics specialist places production or-
ders and monitors their fulfillment;215
• transport logistics specialist places transportationorders and monitors their fulfillment.
C. Challenges of the information service developmentand approaches to facing them
There are several challenges that developers face whencreating integrated systems that are aimed at solving theaforementioned problems:
• systems need to store widely heterogeneous, weaklystructured information, from sensor data to rules andalgorithms which specify system behaviour in caseof contingency situtations;
• systems, on the one hand, should combine multi-ple approaches to information processing, but onthe other hand, production processes are constantlyevolving and thus those approaches should be con-stantly augmented and refined.
• opening new production facilities, structural reorga-nization of the enterprise, product line optimization,etc., all cause major changes to the system; suchchanges should require minimal effort. In otherwords, systems should be highly scalable and flex-ible.
• various devices need to interface with each other,as well as with the information system, yet theyoften have widely different interfaces to the outsideenvironment;
• depending on the user category, task being per-formed, and other factors, there should be multipleforms of representation of the data stored in thesystem;
Ontology-based approach is widely used in softwareengineering[5], [6], as well as in other spheres[7], [8],[9], [10], to solve the problem of heterogeneous infor-mation representation. This approach involves creatinga number of ontologies, at least one for each kind ofinformation being represented in the system. A familyof ontologies created for the information service beingdeveloped as part of this paper will be discussed in thefollowing sections.
Automation industry leaders offer several solutions,which are aimed at building integrated industrial au-tomation and servicing systems. They include "PlantEcoStruxure" from Schneider Electric [11], "Mind-Sphere" from Siemens [12]. Such solutions have thefollowing drawbacks:
• high entry barrier• high cost of ownership• their evaluation versions are limited and/or unavail-
able publicly• even the developers of such systems often cannot
clearly formulate future directions for their systems.
The Internet of Things (IoT) trend also concerns itselfwith software and hardware integration in the hetero-
geneous environments. Its central problems include thefollowing [13]:
• a lot of data is gathered from various devices, butthere are no tools and techniques to analyze themproperly; In other words, the data can be gathered,but then cannot be processed properly;
• creating unified and standardized interfaces betweendevices. The matter is complicated by the fact, thatthe number of interfaces potentially required fordirect integration of heterogeneous devices is pro-portional to the number of device classes, squared.
• security and access control problem.
D. Proposed approach
It is proposed to use OSTIS Technology [14] as a basisfor the proposed approach to solving the problems posed.The basic principles of building a unified informationsystem for activity enterprise automation using this tech-nology are described by the authors in [4]. As part of thisapproach, an enterprise is proposed to be considered asa single information multi-agent system, within which:
• all information is combined into a single informa-tion space (enterprise knowledge base, which isstored in semantic memory);
• all participants in the process (people, robotic sys-tems, various kinds of industrial complexes, etc.) aretreated as agents over this common knowledge base.This means that they (a) monitor the situations ofinterest in the knowledge base and react to them(b) describe the results of their activities in theknowledge base so that this information is availableto other agents and they can analyze it.
• the knowledge base of the system is hierarchicallyorganized, i.e. represents a hierarchy of subjectdomains and corresponding ontologies.
• multi-agent system itself is organized hierarchicallyas well; agents can form arbitrarily deep hierarchies(a collective of agents can have another collective asits member). For example, several separate roboticssystems can be integrated into a robotics complexto perform a particular task.
This approach has a number of advantages in regardsto information services:
• no need to develop tools for direct interaction of sys-tem components (human-robotized system, human-human, etc.) due to their interaction through sharedmemory;
• due to the fact that all agents interact through sharedmemory, in the general case for the system it doesnot matter how this or that agent is physicallyarranged. Thus, the gradual replacement of manuallabor with automated systems or the improvementof such systems does not require making changesto the system;
216
• through the use of a common single knowledgebase and wide possibilities of associative searchin such a knowledge base, any participant in theproduction process at any time has access to all theinformation he needs, and not to any of its prede-termined fragments, the expansion of which maybe related additional overhead. Thus, the variousprocesses monitoring is greatly facilitated and thesearch for answers to questions of interest to usersis accelerated. User requests can be refined in anyway necessary;
• information stored in the knowledge base may beshown differently to different user categories; theinformation itself remains unchanged, only the visu-alization means change. Therefore, there is no needto duplicate the information.
In this work, these principles are implemented in thecontext of solving the problem of information services,while the main focus is on the practical implementationof these principles and demonstration of solving some ofthe problems discussed earlier with concrete examples.
Integrated information service development involvesdecomposing the system into several subsystems aimedat certain particular problems. Nowadays almost everyenterprise implements some or all of these in a traditionalfashion using modern technologies.
Basic manufacturing subsystems include (listed frombottom level up):
1) SCADA performs and facilitates supervisory con-trol, manufacturing data acquisition and archival;
2) MES and WMS• MES (manufacturing execution system) con-
trols product manufacturing process.• WMS (warehouse management) - controls
product storage before shipping out to businesscustomers.
3) ERP (enterprise resource planning) controls, whichproducts will be produced in which quantities andwith which production equipment.
That being said, every subsystem interfaces with bothhigher and lower level subsystems. Therefore, one ofthe tasks in the construction and implementation ofan information service is the integration of existingsubsystems into a single system and making so thatthese systems can be developed independently withoutdisrupting the service as a whole. Ultimately, the con-struction of such a system implies a transition fromthe integration of heterogeneous subsystems to a singleunified technological foundation for the implementationof all the aforementioned subsystems.
According to this approach, information service isconsidered to be an ostis-system (system that is builtupon the OSTIS Technology). The traditional architec-ture of an ostis-system [15] includes an implementation-independent platform for interpreting semantic models
of ostis-systems, and a platform-independent semanticmodel of this ostis-system (sc-model of an ostis-system),which, in turn, includes the following components:
• abstract semantic graphodynamic memory;• semantic model of a knowledge base based on hier-
archical system of subject domains and ontologies[16];
• semantic model of problem solver, which treats itas a hierarchical system of agents controlled byand interacting through the shared semantic memory[17]
• semantic model of an ostis-system interface (includ-ing user interface) which is treated as a kind ofsubsystem, that has its own knowledge base andproblem solver.
[3] paper discussed higher levels of hierarchical sys-tem of subject domains and their respective ontologiesfor ISA-88 formalization. Further work required severaladditions to that system. Several ontologies were builtfor the following subject domains:
• Subject domain of substances• Subject domain of products• Subject domain of personnel• Subject domain of manufacturing situations• Subject domain of contingency situations• Subject domain of logistics situations• Subject domain of warehouse processes• Subject domain of product shippingProblem solver of the information service currently
includes a number of basic search agents, as well asseveral agents that perform more complex tasks, such asidentifying a reason for a certain situation. An exampleof such an agent is discussed below.
Information service interface consists of two parts:• user interface provides access to the required infor-
mation for various categories of end-users;• system interface to existing enterprise automation
subsystems (direct device accesss is not needed,since at this level this information can be obtainedfrom the appropriate subsystem, e.g., a SCADAsystem).
The following are the examples of how the currentimplementation of the system interface works with boththe user and other subsystems, such as SCADA.
II. SYSTEM IMPLEMENTATION
In this section, we consider specific examples of theimplementation of the previously proposed principleswithin the various subsystems of the system being de-veloped.
A. SCADA subsystem implementation
The main task of the information service subsystemrelated to the current implementation of the SCADA
217
system is to provide interactively various reference in-formation about the objects and concepts used in theSCADA system. This subsystem is primarily focused onservicing the foreman and shop manager.
In the current implementation, the principle of oper-ation of the system is as follows: within the SCADA-system interface, there are interactive elements thatuniquely correspond to objects and concepts in theknowledge base of the information service system(currently, connection is established through the mainRussian-language identifier [18]). When a user interactswith an interactive element within the SCADA system, arequest is sent to the information service system contain-ing the identifier of the requested element, after whichthe system displays the semantic neighborhood of therequested element in the current state of the knowledgebase.
Consider the following fragment of the "Khutorok"SCADA. Suppose, a foreman wants to get additionalreference information about the current control recipe.To do this, they click the corresponding button in theproject (see Fig. 1), after which the browser displays theanswer to the query of the semantic neighborhood of theconcept "control recipe" in SCn language (see Fig. 2).
Another way to use the information service systemby the master is to identify the causes of the currentsituation (both standard and non-standard). In the currentversion of the system, the Abstract sc-agent of the searchfor the causes of the current state of a given objectis implemented as part of the problem solver of theinformation service system. The specified sc-agent findsall actions in the knowledge base, as a result of whichthe state of the object that is the query argument has beenchanged.
An example of the operation of this sc-agent is givenin figures 3 and 4. The valve K1Valve2 is part of thecoagulator K1 and is currently open (Figure 3). Afterasking the question, the system, as a result of the sc-agent operation, gives an answer that the valve is open,because the washing operation is currently running forthe coagulator K1 (Figure 4).
B. Logistics subsystem implementation
The main task of logistics subsystems is to ensure theeffective interaction of the actual production, warehouseand transport.
As an example of the work of the logistics subsystem,we consider the task of monitoring the fulfillment of aproduction batch of products.
There are a number of stages associated with thepreparation and delivery of goods batches to the cus-tomer, namely:
• batch production;• cooling the product in a warehouse;• loading the product on the transport;
• product delivery to the customer.Any waiting and downtime (waiting for production,
cooling the product and track downtime while loading)increase the cost of production. Thus, to minimize costsit is necessary, on the one hand, to ensure the minimumdelay between these stages, on the other hand — in theevent of delays quickly change the start times of thefollowing stages.
In turn, the listed stages can be divided into moresimple. For example, batch preparation consists in per-forming one or more recipes on production units (inaccordance with S88), and recipes consist of successivelyperformed operations. If the operation at the time ofexecution for some reason pauses, it means that theproduction time of the recipe is increased. Informationabout this event should go to the knowledge base of theinformation service system, after which they can be usedto automatically or manually assess the criticality of thesituation and, if necessary, adjust further stages.
For example, if the delay in the batch productionprocess exceeds a certain amount, then it is necessaryto postpone the loading time of the batch for deliveryto the customer, otherwise the track will arrive andwait until the product has cooled in the warehouse. Theinformation service system can monitor such situationsand, if necessary, change the time of track departure tothe warehouse for loading, taking into account the newbatch readiness time.
An example of a rule in the knowledge base describingthis kind of adjustment is presented in Figure 5 (it isassumed that the delay should not exceed 30 minutes).
C. The robotic subsystem implementation
The most important component of the informationservice system is the subsystem oriented to work withrobotized systems. According to the above principles forbuilding an enterprise automation system, all participantsin the production process, including robots, are treatedas agents working on a common information space.Thus, on the one hand, robots record the results of theiractivities in the general knowledge base, on the other,they can be controlled by situations and events in theknowledge base. Let us consider in more detail the useof the robotic subsystem on the example of the curdproduction train.
The curd production train currently includes tworobotic nodes: packaging unit for the product in a boxusing a collaborative robot; and transportation unit for apallet with the finished product to the receiving port ofa warehouse based on a mobile robot.
Consider the task of packing curds in boxes by acollaborative robot. General view of the installation isshown in Figure 6.
The facility receives at the entrance a stream of cottagecheese, which is then divided and formed into groups of
218
Figure 1. SCADA Request to the information service
Figure 2. Information service response
219
Figure 3. Relation between valve and coagulator
Figure 4. Valve opening causee
12 pcs. The robot with the help of a gripping mechanismtakes the group and puts it in two boxes. Packing of curdsis allowed in two types of boxes: single-level 6 pcs. andtwo-level 12 pcs. Further, the boxes on the output con-veyor are issued on the palletizing unit. For the operationof the system, the robot needs the following informationreceived from the information service system:
1) Current train performance2) Type of box supplied3) Lack of input boxes4) The state of the production train nodes adjacent to
the robot (Works, does not work, does not work,accident)
5) Service date of the gripping mechanism
Based on this information, the robot software flexiblyresponds to the current parameters of the entire man-ufacturing process. Adjusting the speed and delays inperforming operations, the robot dynamically controlsthe performance of the cell so as not to create queues onthe input and output conveyors. Feed box type determinesthe amount of product that the robot needs to pack. Thestate of the adjacent manufacturing process nodes allowsthe robot to switch the packaging process to manualprocessing mode or to put the product into a temporarystorage device.
Next, we consider the problem of transporting thefinished product to the warehouse using a mobile robot.View of the robot is shown in Figure 7.
The mobile robot operates on the concept of a mission,which consists of a product pick-up point, a delivery pathand an unloading point. After starting the manufacturingprocess, the robot takes the loading position or expectsan external signal that the pallet is full. After that, therobot starts the process of loading the pallets on board.
Further, based on the available room map, the dynamicsituation in the shop in compliance with safety standards,the robot moves with a pallet to the receiving port, where,if there is free space, it unloads.
The construction of the missions for the robot isperformed by the information service system, for thisthe following subproblems are solved:
1) Creating a schedule for exporting a product fromseveral trains, taking into account their perfor-mance and the robot;
2) Scheduling charging times during robot standby;3) Safety level evaluation based on current situation
on the production shop.
D. User interface implementation
The essential feature required of the information ser-vice user interface is that it should be able to visualizeknowledge base information in several ways. Experi-enced users may choose to use universal languages, whileothers should be able to use more specialized languages,that are more suitable to their line of work. As systemfunctionality grows, the need to expand the number ofexternal languages used may arise.
To be able to expand the number of external lan-guages used, OSTIS technology-based computer systemsneed the appropriate interface tools. An approach todeveloping such tools was proposed in the [19] paper.This approach requires three ontology-based models foreach external language: semantic model of languagetexts, syntactic model of language texts, and semantic-syntactic transformation model in the form of ontologyof transformation rules.
Current version of the information service uses twoexternal visualization languages. Procedural models arevisualized using PFC language, and physical models arevisualized using P&ID language. [4] provided severalexamples of formal representation of ontology-basedmodels for PFC language. PFC and P&ID visualizationtools have been introduced in the current version ofinformation service system. Figures 8 and 9 show imagesproduced by the current visualization tools.
Furthermore, to simplify interaction for inexperiencedusers, the simplified visualization mode was developed.It hides parts of the information needed only for theknowledge base developers. Visualization mode, whichshows all the information, is called an "expert mode", andcan be switched on or off with a toggle. Figures 10 and11 show representations of the "procedure" term specifi-cation, as displayed in simplified and expert modes.
III. CONCLUSION
The paper discusses the principles of construction andthe current version of the system of information servicesfor an batch manufacturing enterprise on the example ofJSC «Savushkin Product».
220
Figure 5. Rule describing the adjustment in case of delay in production
Figure 6. Curds laying module
221
Figure 7. Mobile Transport node
Figure 8. PFC visualization example
Figure 9. P&ID visualization example
Development plans of the considered system includethe following urgent tasks:
1) Refusal to edit the source code of the knowledgebase in favor of editing the knowledge base in realtime, which will make the work of users moreefficient and allow employees of the enterprise (forexample, automation engineer) to make changes tothe knowledge base independently.
2) Graphical representation modernization: in the en-gineering documentation, every little thing matters(thickness, color and kinks of the line), in thecurrent implementation the display is somewhatsimplified.
3) Tighter integration of already implemented systemswith the information service system, building aunified system that combines several levels of theenterprise.
ACKNOWLEDGMENT
The authors would like to thank the staff and adminis-tration of the JSC "Savushkin Product" for their supportof the work.
REFERENCES
[1] (2018, Dec.) Integrated industry - industrial intelligence.[Online]. Available: https://www.hannovermesse.de/en/news/key-topics/integrated-industry/index-2.xhtml
[2] J. Stark, Product Lifecycle Management (Volume 1): 21st CenturyParadigm for Product Realisation, 3rd ed. Switzerland: Springer,Cham, 2015.
[3] V. Taberko, D. Ivanyuk, K. Rusetski, D. Shunkevich, I. Davy-denko, V. Zakharov, V. Ivashenko, and D. Koronchik, “Ontology-based design of batch manufacturing enterprises,” in Otkry-tye semanticheskie tehnologii proektirovanija intellektual’nyhsistem [Open semantic technologies for intelligent systems],V. Golenkov, Ed., BSUIR. Minsk : BSUIR, 2017, pp. 265–280.
[4] V. Taberko, D. Ivaniuk, V. Kasyanik, V. Golovko, N. Guliakina,K. Rusetski, D. Shunkevich, A. Boriskin, and N. Grakova, “De-sign of batch manufacturing enterprises in the context of industry4.0,” in Otkrytye semanticheskie tehnologii proektirovanija in-tellektual’nyh sistem [Open semantic technologies for intelligentsystems], V. Golenkov, Ed., BSUIR. Minsk : BSUIR, 2018, pp.307–320.
[5] T. Dillon, E. Chang, and P. Wongthongtham, “Ontology-basedsoftware engineering- software engineering 2.0.” Australian Soft-ware Engineering Conference, IEEE Computer Society, pp. 13–23, 2008.
[6] A. Emdad, “Use of ontologies in software engineering,” 2008,pp. 145–150.
[7] A. N. Andrichenko, “Tendencii i sostojanie v oblasti upravlenijaspravochnymi dannymi v mashinostroenii,” Ontologijaproektirovanija, no. 2(4), pp. 25–35, 2012.
[8] A. Fedotova and I. Davydenko, “Primenenie semanticheskihtehnologij dlja proektirovanija intellektual’nyh sistem upravlenijazhiznennym ciklom produkcii,” Izvestija vysshih uchebnyhzavedenij. Mashinostroenie, no. 3 (672), pp. 74–81, 2016.
[9] A. Fedotova, I. Davydenko, and A. Pfortner, “Design intelligentlifecycle management systems based on applying of semantictechnologies,” vol. 1. Springer International Publishing Switzer-land, 2016, pp. 251–260.
[10] A. Fedotova, V. Tarassov, D. Mouromtsev, and I. Davydenko,“Lifecycle ontologies: Background and state-of-the-art.” Copen-hagen: IARIA XPS Press, 2016, pp. 76–82.
222
Figure 10. Simplified representation
Figure 11. Expert mode representation
223
[11] (2018, Dec.) Ecostruxure is iot-enabled, plug-and-play,open, interoperable architecture and platform, in homes,buildings, data centers, infrastructure and industries. [Online].Available: https://www.schneider-electric.ru/ru/work/campaign/future-of-automation/smart-industrial-automation.jsp
[12] (2018, Dec.) Mindsphere is the cloud-based, open iot operatingsystem from siemens that connects your products, plants,systems, and machines, enabling you to harness the wealth ofdata generated by the internet of things (iot) with advancedanalytics. [Online]. Available: https://www.siemens.com/global/en/home/products/software/mindsphere.html
[13] (2018, Dec.) 7 big problems with the internet of things. [Online].Available: https://www.cmswire.com/cms/internet-of-things/7-big-problems-with-the-internet-of-things-024571.php
[14] V. V. Golenkov and N. A. Guljakina, “Proekt otkrytojsemanticheskoj tehnologii komponentnogo proektirovanijaintellektual’nyh sistem. chast’ 1: Principy sozdanija,” Ontologijaproektirovanija, no. 1, pp. 42–64, 2014.
[15] V. Golenkov, N. Guliakina, N. Grakova, I. Davydenko, V. Niku-lenka, A. Eremeev, and V. Tarassov, “From training intelligentsystems to training their development tools,” in Otkrytye seman-ticheskie tehnologii proektirovanija intellektual’nyh sistem [Opensemantic technologies for intelligent systems], V. Golenkov, Ed.,BSUIR. Minsk : BSUIR, 2018, pp. 81–98.
[16] I. Davydenko, “Semantic models, method and tools of knowledgebases coordinated development based on reusable components,”in Otkrytye semanticheskie tehnologii proektirovanija intellek-tual’nyh sistem [Open semantic technologies for intelligent sys-tems], V. Golenkov, Ed., BSUIR. Minsk : BSUIR, 2018, pp.99–118.
[17] D. Shunkevich, “Agent-oriented models, method and tools ofcompatible problem solvers development for intelligent systems,”in Otkrytye semanticheskie tehnologii proektirovanija intellek-tual’nyh sistem [Open semantic technologies for intelligent sys-tems], V. Golenkov, Ed., BSUIR. Minsk : BSUIR, 2018, pp.119–132.
[18] (2018, Nov.) IMS metasystem. [Online]. Available: http://ims.ostis.net/
[19] A. Boriskin, D. Koronchik, M. Sadowski, I. Zhukov, and A. Khu-sainov, “Ontology-based design of user interfaces,” in Otkry-tye semanticheskie tehnologii proektirovanija intellektual’nyhsistem [Open semantic technologies for intelligent systems],V. Golenkov, Ed., BSUIR. Minsk : BSUIR, 2017, pp. 265–280.
ПРИНЦИПЫ ПОСТРОЕНИЯ СИСТЕМЫКОМПЛЕКСНОГО ИНФОРМАЦИОННОГО
ОБСЛУЖИВАНИЯ СОТРУДНИКОВПРЕДПРИЯТИЯ РЕЦЕПТУРНОГО
ПРОИЗВОДСТВА
Таберко В.В., Иванюк Д.С.,Касьяник В.В., Головко В.А.,
Русецкий К.В., Шункевич Д.В., Гракова Н.В.
В данной работе предлагается продолжение при-менения онтологического подхода к проектированиюпредприятий рецептурного производства. В рамкахпродолжения формализации стандартов рассматрива-ется графические представления внешнего языка спе-цификации процедурных моделей Procedure FunctionChart и P&ID-схем (стандарт ISA-88), дающие инжене-рам привычный инструмент для работы. Реализованыагенты получения дополнительной информации и ана-лиза типовой нештатной ситуации. Рассматриваетсятакже агентно-ориентированный подход к организа-ции взаимодействия роботов в рамках роботизиро-ванных производственных комплексов, основанный навзаимодействии через общую семантическую память.
Received 10.01.19
224
Examples of the use of artificial neuralnetworks in the analysis of geodata
Valery B. TaranchukBelarusian State UniversityMinsk, Republic of Belarus
Abstract—The article discusses the problems of devel-opment, tool filling, and usages of the integrated programcomplex of the composer of digital geoecological models.Possibilities of interactive graphics visualization, and com-parison of results are marked. The results of application ofartificial neural networks in the analysis and interpretationof geospatial data are presented and discussed.
Keywords—digital geoecological model, approximatingdigital field, computer algebra system Mathematica, ge-ographic information system Surfer, interactive graphicsvisualization, artificial neural network
I. INTRODUCTION
The construction of the digital geoecological, geolog-ical models at present is a compulsory part of expertiseto many areas. Geological simulation is an independentstream, which includes the progressing of mathematicalmethods and algorithms; development of computer pro-grams, which provide the cycle of models’ construction,database creation, provisioning and maintenance. Thecorresponding geo-environmental information productsinclude the loading from different sources and datapreprocessing, correlation, creation of digital cubes ofreservoir properties, interactive data analysis, visualiza-tion with the help of any type graphics, mapping. Thereare a lot of software tools for this purpose. As a rule,the corresponding software packages are focused onsolving a specific class of problems. One of the activelydeveloping new directions in Geology and Geoecologyis the improvement of computer systems based on theintegration of powerful tools of spatial and temporaldata processing offered by GIS with models of artificialintelligence of neural networks [1] – [4]. Different ap-plications are created that allow not only to implement ahuge number of algorithms for data processing, analysisand visualization, but also provide opportunities to obtainnew results.
On the other hand, it can be noted, that while gen-eral issues are being resolved and there is no properelaboration of details. Current achievements and toolsof intellectual (symbolic) calculations are not fully used,a number of performed products are not justified bythe theory of numerical methods. Most of the availablepackages lack tools to assess the accuracy of the results.Any extension of functionality requires the involvement
of highly qualified programmers. Therefore, an importantdirection is the development of software systems basedon the combination and integration into a single environ-ment of computer algebra systems (CAS) and geographicinformation systems (GIS). Moreover it should be con-sidered that to solve the problem of processing initial datathere’s no specific GIS to be the full set of space-analyticmethods and analysis tools. In many cases it’s necessaryto combine the tools provided by GIS with programsfor static data analysis, tools for mathematically complexcomputations which include implementations of modernmethods and algorithms of analysis and interpretation ofspatial data. A number of methodological and technicalsolutions aimed at overcoming the mentioned difficultiesare proposed and implemented in the integrated computersystem GGMD – “The generator of the geological modelof deposit” [5] – [7].
II. THE DEVELOPMENT PLATFORM OF GGMD
GGMD is assigned for creation and estimated accuracyof configurable geological model based on the usageof CAS and GIS, “smart” methods of model adapta-tion while in service, “self-tuning” of models consid-ering additional data from the actual development ofprocesses. Development platform is computer algebrasystem Mathematica [8], language is Wolfram Language[9], geographical information system is Golden SoftwareSurfer [10]. While programming in Wolfram Languagetechnical solutions, described in [11], were implemented,moreover software system in a particular configurationcan be used after it’s built and saved in computabledocument format [12]. Calculations, user work withCDF version of application are possible on every per-sonal computer. When viewing CDF version, hosted onwebserver, viewer is automatically loaded in the formof browser plugin. Offline work is possible after theinstallation of free distributed CDF Player.
Let’s make some clarifications for the illustrations ofimplementations marked above, the results of the usageof program modules which are selected and modified forthe problems we’re solving. Below we mention the com-ponents, which are actually standalone program modules.They can be also considered as parts of automated
225
workstation of specialist, who during the computationalexperiments works out techniques of adaptation of digitalfields. We should specifically note an important technicalsolution – all the work steps with the complex of modulesare provided with the possibility of import and export ofobtained result with several configurations of output for-mat [13]. It provides the user with additional possibilitiesfor performing similar calculations in different (includingthe others) applications, the comparison of results.
It is necessary to understand that creation and main-tenance of geoecological model don’t expect to haveunique solution to a mathematical problem. Subjectiveopinion, the qualification of an expert – are the factorsthat always take place in such activity. While workinga user has to operate with data of different accuracy,some initial data is even conflicting; data density withmeasurements differs on different parts. That’s why forconstruction digital models it’s important to have toolsfor interactive data processing, simulation of possiblesituations of receiving and correction of input data.All the steps of working with data in GGMD includevarious options of graphic visualization [14], logging andcomparison of incoming and placed to archive results.Complex’s tools give a user possibility to “play” withinitial data and compare the results with prepared etalons,what is more it’s allowed to import and export the dataand images and to scale them. Extensive data exchangepossibilities are important for simultaneous work inseveral software environments.
III. THE COMPONENTS OF GGMD
In computer system GGMD the following tools areimplemented:
• tools and patterns for preparation of reference modelof digital field, which corresponds to the specifiedproperties (“Digital field constructor”);
• tools and several options of “distortion” of referencemodel;
• tools for data capture simulation, which are usedin simulation practice (“Generator of profile ob-server”);
• modules for calculation, visualization, comparisonof digital fields approximation by several differentmethods (“Approximation component”);
• tools and adaptation modules for digital modelbeing formed (“Adaptation component”).
The main idea and purpose of the development ofthis computer system is to choose the method of pro-cessing the original data by comparing the referencedigital field and reconstructed by "observations". Thereference distribution for a rectangular area is formedusing mathematical descriptions. Each expert determinesand includes own typical fragments in the model. Thenthe instruments of the GGMD perform "observations",simulate the measurement of the reference distribution,
and the geometry of the measurement points and theiraccuracy are also determined by the user of the systemand should approximately correspond to the initial dataof the of the field of interest. The layout of points withmeasurements should not be regular. As a result of thisstage, the user receives a set of data "observations",the main ones being the coordinates of the point andthe value in it. The next step is to select the algo-rithm for processing the resulting set ("reproducing" thedigital field) by performing interpolation and extrapola-tion. Comparison of the results of "playback" and thereference model will prompt the expert the method ofprocessing, the geometry of the observation points.
A. Digital field constructor (DFC). Base surface forma-tion
Software components from this group provide in in-teractive mode the construction of the model’s surfacefrom standard elements with accompanied visualizationof mathematical description (analytic function), model’ssurface is interpreted as a relief – set of surface shapes.The construction is made in the module which is pro-grammed in system Mathematica and includes the gen-eration of surface equation – function of two argumentsx and y which is continuous (or piecewise continuous)and defined in the rectangle. User defines the boundariesof domain xMin and xMax, yMin and yMax and surfaceheight limits zMin and zMax. Let’s mark out that allthe notations in DFC are given in format of InputForm(string format), that is accepted specially as some userscan use application written in Excel, Delphi, C or oth-ers, where mathematics notation isn’t supported insidethe program code. There are mathematical expressions(elements) which allow for the user of the GGMD toreproduce the behavior of the areas, which are typical forrelief, in the set (library) of components of the functionbeing formed. User at the first stage of reference modelformation sets up a piecewise-defined function zBasic(x).In the terminology of the complex, this is the basic profile– tape of specified width and length, which imitates thetypes of relief with the elements of plateau, slope, cliff.
Different variants of determining the basic profile ofthe base surface are given in [5] – [7]. We note thatthe user of the system obtains an analytic expression.Analytical expressions (1) are used in the preparation ofthe following examples:
zBasicQ(x) =If [xMin ≤ x ≤ xOtkQ1, fP ltQ(x, 0), 0]+
+If [xOtkQ1 < x ≤ xOtkQ2, fOtkQ(x, 0), 0]++If [xOtkQ2 < x ≤ xSklQ2, fPrhQ(x, 0), 0]++If [xSklQ2 ≤ x ≤ xMax, fSklQ(x, 0), 0],
(1)
fP lt(x, y) = zP lt, fOtkQ(x, y) = Tan(ugOtkQ) ·(x − xOtkQ1) + zP ltQ, zOtkQ2 = fOtkQ(xOtkQ2, 0),fbfQ(x, a) = D(fOtkQ(x, y), x) − 2 · a · (x − xOtkQ2),perkoefQ = afQ/.Solve[fbfQ[xSklQ2, afQ] ==Tan(ugSklQ), afQ][[1]], fPrhQ(x, y) = fOtkQ(x, y) −perkoefQ · (x − xOtkQ2)2, fSklQ(x, y) = Tan(ugSkl) ·
226
(x − xSklQ2) + zSklQ2, zSklB = fPrhB(xSklB2, 0),zOtkB2 = fOtkB(xOtkB2, 0).
Further 2 surfaces are considered. For the upper in-stead of the letter Q should substitute T in expression (1),for the lower – B. xOtkQ1, xOtkQ2 – are coordinatesof transition points “plateau – slope”, “slope – slightslope”, ugOtkQ, ugSklQ defines the inclines of slope,slight slope, zPlt=10. The constants in expression (1):xOtkB1 = 40, xOtkB2 = 70, xSklB2 = 75, zPltB = 10,ugOtkB = 1.1, ugSklB = 0.15, xOtkT1 = 30, xOtkT2 =45, xSklT2 = 66, zPltT = 10, ugOtkT = 1.2, ugSklT =0.08. (see Fig. 1).
Figure 1. Plot of basic profiles.
In the given example the base surface model is quasithree dimensional (the level of z doesn’t depend on y).Basic surface (tape) is made up of 3 typical sectors: flathorizontal (plateau), flat with fast rising level (slope),flat with slow rising level (slight slope). The connectionbetween the sectors is continuous. Transition “plateau -slope” is made at a selected angle, transition “slope -slight slope” is smooth.
Then user can add perturbations of different shapes,sizes and orientations to the base surface. The construc-tion with DFC module of basic profile from the frag-ments is possible with continuous transition “fragment –added fragment”, smooth transition, a jump (imitation ofsplit).
B. Digital field constructor. Reference surface formation
The next step of construction with DFC module isusage of program module’s tools to add perturbations,fragments of typical elements of relief to a basic pro-file. Template (patterns) library includes elements whichcorrespond to perturbations (areas of distortion of basicsurface) of different geometrical shape. While connectingthe patterns it’s possible to set interactively their positionand size. Described mathematical elements, which imi-tate the following shapes of relief: hill, embankment, pit,excavation, trench, canal, quarry, ravine, hollow (vug),are included in basic package ( [5], [6]). It should benoted that all the elements listed above are specified byanalytic expressions, such as z = fFrgm(x, y). Besidesit, those are written for the square [-1,1]×[-1,1], and then
in the final function the arguments are scaled. SystemMathematica includes a big amount of spatial graphicprimitives of which cone, ball, cuboid, cylinder are usedin DFC module, also different pyramids are includedto the library. Several variants of determining of thereference surface are given in [6], [7].
An example of reference surface model formation,which is obtained from the basic surface, by addingelements of the listed types (2 pyramids, 5 hills) isshown below on Fig. 2. It’s important that in the re-sulting equation (2) the coefficients in the formulas ofperturbation elements fHill, fPyramid are chosen by userwhile visual construction. The constants in expression(2) for zSurfB(x,y) are fh2Q=fh2B=10, fh5Q=fh5B=20.
fOriginB(x, y) = zBasicB(x).zSurfB(x, y) = fOriginB(x, y)+
+18 · fPyramid1(0.12 · (x− 41), 0.18 · (y − 52))++20 · fPyramid1(0.12 · (x− 20), 0.15 · (y − 10))+
+20 · fHill(0.1 · (x− 12), 0.1 · (y − 40))−−fh2Q · fHill(0.12 · (x− 88), 0.1 · (y − 15))−−12 · fHill(0.06 · (x− 77), 0.08 · (y − 45))++12 · fHill(0.08 · (x− 45), 0.07 · (y − 28))++fh5Q · fHill(0.1 · (x− 12), 0.1 · (y − 40)).
(2)
Figure 2. Plot of surface zSurfB.
An example of reference surface model 2 is shownbelow on Fig. 3. The constants in expression (2) forzSurfT(x,y) are fh2Q=fh2T=8, fh5Q=fh5T=26.When programming the module DFC, the followingfunctions of the system Mathematica were used: Solve,Piecewise, Cases, Table, Plot, Plot3D, Manipulate. Whilevisual examinations of the plots in DFC module, userhave a possibility to define the coefficients of functionexpression by moving sliders or setting specific valueson the panels which are the part of the interface ofManipulate function (more in [14], [11]). At this stage ofconstructing of the surface, the user of the DFC modulealso obtains an analytic expression.
C. Generator of profile observerMore than ten options of 1D, 2D and 3D plots were
implemented in GGMD system, including modules for227
Figure 3. Plot of surface zSurfT.
obtaining and designing maps and inserts on them,plots on profiles, 3D visualizations of possible incisions(vertical, horizontal), clipping (simple and complex). Theillustrations of the usage of several visualization tools aregiven in [5], [7].
Let’s consider the examples of obtaining and con-figuring of graphs of geological profiles with using ofthe tools of “Generator of profile observer” (GPO) –the illustrations are given on Fig. 4, Fig. 5 for surfaceszSurfT, zSurfB on direction (0,45) – (100,10).
Figure 4. Example of indicating of profiles, graphics of cross-sections.
While solving the problems of the geological simulation,profiles are used to visualize the connection betweenrelief and the structure of Earth’s crust. Geological profileis a graphic image in vertical plain of subsoil structureand deposits contained in it. In our case the profile – isa line, which is obtained at the intersection of analyzedsurface and vertical surface in a given direction.
We will consider the use of GPO tools to showseveral methods of simulating observations and obtaininginitial data by instruments of the GGMD’s module of“distortion” on the example of profiles on referencesurfaces.
Figure 5. Example of the simple incisions of surfaces zSurfB, zSurfT.
IV. EXAMPLES OF ANALYSIS OF GEODATA BY TOOLSOF NEURAL NETWORKS
The problems of construction of geological modelscertainly belong to the class of complex ones. Andfor the solution of complex problems it is required toprovide compatibility and integration of the differentmodels of data representation and knowledge process-ing algorithms. Approaches may be different, some areconsidered in [15]. Wolfram Mathematica, as a systemof intelligent computing, provides the user with not onlythe means of mathematical transformations, accurate andapproximate calculations, but also the tools of machinelearning. They may be used in the interpretation andprocessing of input data and simulation results. Theresults and examples of the analysis of geoecologicaldata by means of neural networks are obtained using thecorresponding functions, which are available in version11 of CAS Mathematica [8], [16]. The results below areillustrated by examples of interpretation and processingof data for profiles of zSurfB, zSurfT surfaces.
A. Preparation of data for numerical experiments
Let’s consider the examples of obtaining and config-uring of graphs of profiles with using of the tools of“Generator of profile observer”. Figure Fig. 6 shows thegraphs of the levels of surfaces zSurfT, zSurfB at theprofile on direction (0,45) – (100,10). Symbols on thegraphs (filled circles and triangles) are marks of surfacelevel values at profile points (regular grid, constantstep). These values are calculated using formulas fromsurface equations. Circles and triangles indicate valuesthat simulate measurements. For the surface zSurfTthey are obtained by adding “noise” using generatorRandomVariate[NormalDistribution[0, 6]], such gen-erator gives a pseudorandom variate from the symbolicdistribution dist. NormalDistribution[µ,σ] represents aGaussian distribution with mean µ and standard devia-tion σ). For the surface zSurfB distortion obtained byadding “noise” using the generator RandomReal[-10,10]
228
on the entire profile, RandomReal[xmin, xmax] gives apseudorandom real number in the range xmin, xmax.
Figure 6. Data for numerical experiments.
B. Results of application of artificial neural networks
Note that we will process data with “distortions” fortwo completely different purposes, respectively, differenttools of neural networks will be used.
In solving problems of mathematical modeling, theoriginal equations are written in differential form, sothe original data (tooling of model) must be continuous,moreover, as a rule, the distribution should be smoothfunctions (for example, see [17]). In other words, thedistribution of the observed parameter along the profileshould be transferred to the initial data of the computermodel as a smooth function. Note that the originaldata of figure Fig. 6 the requirement of smoothness isresponsible, and data from the “distortions” is unsuitablefor numerical models, this baseline data will immediatelylead to “bumpy” decisions (computational instability)and the results will be unusable.
Examples of obtaining a “smooth” profile.The results of obtaining in the module “Adaptation”
of GGMD complex of smoothed profiles are presentedin figures Fig. 7, Fig. 8.
Figure 7. Smoothed profile for zSurfT. Method – RMSProp.
The functions and the options of the neural networkfor Fig. 7 are next: netA = NetChain[vectLength, Tanh,
Figure 8. Smoothed profile for zSurfB. Method – ADAM.
vectLength, Tanh, 1, ...]; netA1 = NetTrain[netA, dat-aProfT1, Method – "RMSProp"], vectLength = 300.Method RMSProp is stochastic gradient descent usingan adaptive learning rate derived from exponentiallysmoothed average of gradient magnitude [18].
For Fig. 8 – vectLength = 25, Method – ADAM,stochastic gradient descent using an adaptive learningrate that is invariant to diagonal rescaling of the gra-dients.
The results of the neural network application for thetasks to be solved in the interpretation and analysis ofgeodata.
Figures Fig. 9 and Fig. 10 show the results of using aneural network for a completely different kind of prob-lem – problems that need to be solved in the interpreta-tion and analysis of observational data, measurements ofparameters ( (for example, see [19])).
Such tasks can be explained in simple words – it isknown that the data to be analyzed contain measurementerrors, including system and random ones. System errorsare “separated” taking into account the knowledge oftheir nature and behavior, and random errors are decidedby experts.
In this consideration, the task is to train the system to“clean” the data from random noise (extract and discard).Since we know how the data illustrated on the graph wereprepared, we can interpret the original set as a standard,but the program (algorithm, executive software module)does not “know” the original.
The illustrations show two different approaches whensetting up a neural network. In the variant of Fig. 9the following condition is accepted – the processeddata set contains 2 parts, one of which is the maindistribution, and the other is random noise. Infor-mation about the nature of the noise in the rulesof training is not given. The results of the calcula-tions in figure Fig. 9 are obtained with the follow-ing parameters: dataAT2=RandomSample[dataProfT2];vectLength=300; netAT2 = NetChain[vectLength, Tanh,vectLength, Tanh,1,..]; trainAT2,testAT2 = Take-Drop[dataAT2,36]; netAT2 = NetTrain[netAT2, trainAT2,
229
Figure 9. Example 1, zSurfT – “clean” the data from random noise.
Figure 10. Example 2, zSurfB – “clean” the data from random noise.
ValidationSet→testAT2] ( [20], [21]).Figure Fig. 10 shows the results of data processing for
profile on zSurfB. The figure illustrates the comparisonof the source and the result when the program “cleared”the data from random noise. Other functions of neuralnetwork configuration are used: NetTrain[netAB1, dat-aProfB2, Method→"ADAM", "L2Regularization"→0.1].
V. CONCLUSION
The article discusses the problems of development,tool filling, and usages of the integrated program com-plex of the composer of digital geoecological mod-els. Possibilities of interactive graphics visualization aremarked. The presented results, examples of processingand visualization of spatial data, the noted methods oftuning of artificial neural networks tools are confirmationof the wide possibilities of the considered technology.On the other hand, the presented methodological andtechnical solutions for the generation and filling of aspecialized software complex indicate the need to createintelligent medium of automatic connection of computertools, the formation of a knowledge base with sets ofreference and standard examples.
REFERENCES
[1] Bryan C Pijanowski, Daniel G Brown, Bradley A Shellito, Gaurav AManik, “Using neural networks and GIS to forecast land use changes:a Land Transformation Model”, Computers, Environment and UrbanSystems, vol. 26, Issue 6, November 2002, pp. 553–575.
[2] Vincenzo Barrile, Giuseppe M.Meduri, Giuliana Bilotta, Ugo MonardiTrungadi, “GPS- GIS and Neural Networks for Monitoring Control,Cataloging the Prediction and Prevention in Tectonically Active Areas”,Procedia - Social and Behavioral Sciences, vol. 223, 10 June 2016, pp.909–914.
[3] SONG Lirong, ZHAO Shiwei, LIAO Weilin, WANG Zhaoli, “NeuralNetwork Application Based on GIS and Matlab to Evaluation of FloodRisk,” International Conference on Remote Sensing, Environment andTransportation Engineering (RSETE 2013), pp. 296–299.
[4] Alper Sen, M. Ümit Gümüsay, Aktül Kavas, and Umut Bulucu, “Program-ming an Artificial Neural Network Tool for Spatial Interpolation in GIS - ACase Study for Indoor Radio Wave Propagation of WLAN,” Available at:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3705544/ (accessed 2018,Dec).
[5] V. Taranchuk, “The integrated program complex of the composer ofgeological models. The concept, solutions”, Computer Algebra Systemsin Teaching and Research, vol. VI, 2017, pp. 186–194.
[6] V. Taranchuk, Viktoryia Taranchuk, “The integrated program complexof the composer of geological models. Examples of results”, ComputerAlgebra Systems in Teaching and Research, vol. VI, 2017, pp. 195–203.
[7] V. Taranchuk, “The integrated computer complex of an estimation andadapting of digital geological models”, Studia i Materiały, N 2 (14), 2017,pp. 73–86.
[8] WOLFRAM MATHEMATICA. Available at: http://www.wolfram.com/mathematica/ (accessed 2018, Dec).
[9] S. Wolfram, An Elementary Introduction to the Wolfram Language. Avail-able at: http://www.wolfram.com/language/elementary-introduction/2nd-ed/ (accessed 2018, Dec).
[10] Surfer. Explore the depths of your data. Available at: https://www.goldensoftware.com/products/surfer/ (accessed 2018, Dec).
[11] Taranchuk V.B. “Osobennosti funkcional’nogo programmirovaniya inter-aktivnyh graficheskih prilojenii”, Vestnik Samarskogo gosudarstvennogouniversiteta. Estestvennonauchnaya seriya, razdel Matematika, N 6 (128),2015, pp. 178–189, (in Russian).
[12] CDF. Documents Come Alive with the Power of Computation. Availableat: https:/www.wolfram.com/cdf/ (accessed 2018, Dec).
[13] Importing and Exporting Data. Available at: https://reference.wolf-ram.com/language/tutorial/ImportingAndExportingData.html/ (accessed2018, Dec).
[14] Annotating & Combining Graphics. Available at: https://reference.wol-fram.com/language/guide/AnnotatingAndCombiningGraphics.html/(accessed 2018, Dec).
[15] V. Golenkov, N. Guliakina, I. Davydenko, D. Shunkevich, “SemanticModel of Knowledge Bases Representation and Processing”, [Electronicresource] // CEUR Workshop Proceedings. Moscow, 2017. Available at:http://ceur-ws.org/Vol-2022/paper51.pdf (accessed 2018, Dec).
[16] Neural Networks. Available at: https://reference.wolfram.com/language/guide/NeuralNetworks.html (accessed 2018, Dec).
[17] A. Chichurin, H. Shvychkina “Computer simulation of two chemostatmodels for one nutrient resource”, Mathematical Biosciences, 278, 2016,pp. 30–36.
[18] NetTrain. Available at: https://reference.wolfram.com/language/ref/ Net-Train.html (accessed 2018, Dec).
[19] V. Orlov, E. Detina, O. Kovalchuk “Mathematical modeling of emergencysituations at objects of production and gas transportation”, MATEC Webof Conferences IPICSE-2018. V. 251, 04059.
[20] TakeDrop. Available at: https://reference.wolfram.com/language/ref/ Take-Drop.html (accessed 2018, Dec).
[21] ValidationSet. Available at: https://reference.wolfram.com/language/ref/ValidationSet.html (accessed 2018, Dec).
ПРИМЕРЫ ИСПОЛЬЗОВАНИЯ НЕЙРОННЫХСЕТЕЙ В АНАЛИЗЕ ГЕОДАННЫХ
В. Б. Таранчук
В статье обсуждаются вопросы разработки, инстру-ментального наполнения и использования интегриро-ванного программного комплекса тестировщика циф-ровых геоэкологических моделей. Отмечены разныевозможности интерактивной графической визуализа-ции и сопоставления результатов. Представлены иобсуждаются результаты применения искусственныхнейронных сетей в анализе и интерпретации геодан-ных.
Received 01.01.19230
Semantic Technology of IntellectualGeoinformation Systems Development
Sergei SamodumkinBelarusian State University of Informatics and Radioelectronics
Minsk, Republic of [email protected]
Abstract—This paper is devoted to the creation of aspecial technology for designing intelligent geo-informationsystems that use knowledge of terrain objects to solveapplied tasks in problem areas.
Keywords—semantic technology, geographic informationsystem, topologic-geographical relations
I. INTRODUCTION
Knowledge and data about terrain objects can interestus not only as spatial data and knowledge, moreoverthey are an integration basis for various subject domains.Formalization of such knowledge and its presentation inknowledge requires both determination of subject rela-tions for description of properties and consistent patternsinherited in observing subject domain using terrain ob-jects, but also determination of geometric characteristics,that are able to bind objects on area. In addition, taking inaccount life of information and terrain object themselves,there is possibility of retrospective analysis, which makespossible to observe terrain objects in subject domainsnot only from their spatial position and and semanticattributes, but also to take into account temporal aspectof existing of terrain objects. As intellectual systemsare designed to meet information needs of users, theseprerequisites promote an expansion of subject domainsand an addition of new functionality in the design ofintelligent geographic information systems [1-6, 11].
However, until now, knowledge about terrain objectswas considered as cartographic data and was a result ofmapping of search queries with applying result to terrainmaps. At the same time, due attention was practicallynot given to the data on the terrain as an integratingelement of different subject domains, the dynamics oftheir change, taking into account their actualization intime counts. Well-known studies aimed at ensuring con-sistent information exchange between spatial and subjectknowledge to ensure, that semantic interoperability wereconducted for systems based on RDF, RDFS, OWLSemantic Web technology stack and as shown in [7]the OWL web language presents advanced capabilities todescribe the subject domains of interacting systems andgives machine-interpretable definitions of fundamentalconcepts of subject domain and the relationship betweenthose in the ontology. However, formalization of the
subject domains and ontological engineering is onlyone of stages in the design technology of intelligentsystems and are not sufficient in themselves to implementconclusions based on knowledge, because ontologicalengineering allows describing declarative knowledge ofsubject domain, while procedural knowledge allows de-signing task solvers and implementing knowledge basedconclusions.
The availability of developed design technology andtools is an important point that reduces, on the one hand,period of development of intelligent systems, and on theother hand, increases functionality of intelligent systemsthat use knowledge of objects of terrain as an integration.At the same time, the design technology should ensurereusing of information and functional components of thesystem in order to shorten the design and developmenttime of the applied systems. The above requirementsare possessed by the semantic technology for designingintelligent systems — OSTIS, the advantage of which isits extensibility both in terms of expanding the subjectdomains, types of knowledge used and their formaliza-tion, and in terms of functionality.
Despite the advantages of the OSTIS [8] technology,knowledge of the objects of the terrain as integrating el-ements, the semantic interoperability of such knowledgeis not paid attention to, the procedures for interaction ofspatial knowledge with knowledge of subject domains,and the possibility of correlating objects of terrain in timehave not been established. Thats why actual task is, first,to design spatial ontologies taking into account the lifecycle of terrain objects and based on their solution tothe problem of semantic interoperability of knowledgeof subject domains, as well as solving the problem ofmanaging metadata and improving search, access andexchange in the context of growing volumes of spatialinformation and services provided by numerous sourcesof geo-information, secondly, the implementation of con-clusions based on knowledge using spatial and math-ematical information as components of knowledge ofterrain objects; thirdly, the integration of the cartographicinterface as a natural way of presenting information aboutterrain objects to humans.
This work is devoted to the creation of a private231
technology of designing intelligent systems that useknowledge about objects of the terrain to solve appliedproblems in subject domains. In this connection, thepaper provides
1) a formal basis for representing knowledge aboutterrain objects taking into account the spatial com-ponent (cartographic knowledge description lan-guage) and formalizing the thematic componentof terrain objects based on ontological engineeringof subject domains using knowledge about terrainobjects;
2) the organization of conclusions based on knowl-edge using knowledge about the objects of theterrain;
3) integration of the cartographical interface.Private technology of designing intellectual geo-
information systems considered in the work is basedon the semantic component technology of designingintellectual systems. Graph-dynamic models of a specialkind - semantic models of knowledge representation andprocessing based on semantic networks - are used as aformal basis of the projected intellectual systems and thebasis of abstract logical-semantic models of intellectualsystems.
II. ONTOLOGY OF TERRAIN OBJECTS AS A FORMALBASIS FOR INTEGRATING KNOWLEDGE OF SUBJECT
DOMAINS IN THE DESIGN OF INTELLIGENTGEOGRAPHIC INFORMATION SYSTEMS
In geographic information systems (GIS), terrain data,which is called geo-information data (geodata), is con-sidered as the basis for solving specific applied problems.To solve such problems, it is proposed to use ontology asa conceptual model that allows to represent objects of aterrain at the semantic level and at the knowledge level.The advantage of using ontologies is that the describedspatial data in the form of a semantic code, formallyinterpreted (understood) by a computer, can also ensurethe integration of geodata obtained from various sourcesand in various forms of representations. In addition,different experts in different subject domains (for theimplementation of interdisciplinary tasks) completelydifferently represent the same data sets in GIS. First ofall, it is connected with different cartographic coordinatesystems, which can be reference (that is, adapted to aspecific part of the Earth’s surface) and common, useellipsoids with different parameters and, accordingly,have different coordinates of the same physical objectof the terrain. Thus, when presenting spatial objects,there is semantic inconsistency, for eliminating whichit is necessary to perform ontological engineering andidentify key objects of subject domains that use spatialdata to represent terrain objects taking into account thelife cycle of terrain objects as well as typical relationshipsover terrain objects.
The selection for each of the classes (types) of terrainobjects of the main, inherent only to him, semanticcharacteristics is the basis of ontological engineering ofterrain objects. At the same time, metric characteristicsof terrain objects do not possess this property. To indicatethe semantic properties of the terrain feature classes, itis proposed to use the topographic information classifierdisplayed on topographic maps and city plans NCRB012-2007 [9].
According to the that classifier, each terrain object hasunique unequivocal designation. The hierarchical classifi-cation has eight grading stages: class code, subclass code,group code, squad code, suborder code, species code,subspecies code. Thus, thanks to the coding of alreadydefined generic connections, reflecting the interrelationsof various classes of terrain objects, the characteristics ofclasses of terrain objects are also established. Due to thefact that these properties and relations are not relatedto specific physical objects, but to their classes, thismeans that specific meta-information objects are a set ofthese meta-information of an intellectual geo-informationsystem.
The ontology of terrain objects includes the descriptionof the following classes of terrain objects:
– water objects and hydrotechnical structures;– settlements;– industrial, agricultural and socio-cultural objects;– road network and road structures;– vegetation cover and priming.The ontology of terrain objects is a classification tree
in accordance with the hierarchy shown in Figure 1. Foreach class of terrain objects, generic links are established.For instance, Figure 2 shows the hierarchy of waterobjects.
Figure 1. Levels of hierarchy for classes of terrain objects
For each class of terrain objects, semantic features thatcharacterize terrain objects are set. At the same time, foreach class of terrain objects, its own characteristic set offeatures is highlighted (for example, in Figure 3, for allobjects of the type "river", the relations "eigenvalue*","width on a scale*", "sign of shipping*", "water qualityfeatures*").
Thus, the considered ontology of terrain objects andthe method of its formal setting allow us to describe allthe main classes of terrain objects and establish for these
232
Figure 2. Hierarchy for terrain water objects
Figure 3. Assignment of semantic features for terrain object «river»
classes a set of features characteristic of the consideredclass of terrain objects.
At the same time, the designed ontology of terrainobjects is not enough to provide conclusions based onknowledge, since it is required to set relationships overterrain objects. The representation of terrain objects ina graph-dynamic model required the identification oftypes of relationships that can be defined over terrainobjects. In addition to subject connections for processingknowledge in intelligent geographic information sys-tems, it is necessary to set topological and geographicalrelations. To this end, we select the types of terrainobjects: areal objects, linear, polylines and points. TableI shows all established relationships, the types of objectsfor which they are established, as well as a schematicrepresentation of the relations and structures for theirstorage in the language of semantic networks used inOSTIS technology.
Table ITOPOLOGICAL RELATIONS, SET ON TERRAIN OBJECTS
Relation type Object type Scheme SC notation
InclusionAreal and linear(multilinear) ob-jects
Areal objects andpoints
Areal objects
Bordering Areal objects
Intersection Linear (multilin-ear) objects
Areal and linear(multilinear) ob-jects
Contiguity Linear (multilin-ear) objects
III. CONCLUSIONS BASED ON KNOWLEDGE USINGSPATIAL AND THEMATIC INFORMATION
The implementation of the findings, based on knowl-edge, is implemented by domain solvers using agents[10]. At the same time, in connection with the specifics ofgeographic information systems, operations for workingwith topological-geographical relations, operations forsemantic comparison of objects of terrain, and integrationof subject solvers are proposed.
A. Operations with topologic-geographical relations
The search for the inclusion relationship is possiblebetween areal, areal and point, areal and linear (multilin-ear) objects. As particular cases - the search for relationsof inclusion between administrative objects (regions,districts, settlements). To solve this problem, algorithmsof computational geometry are used. In general cases, thedefinition of an inclusion relationship between objectsof a locality is reduced to an algorithm for determiningwhether a polygon contains a point. For areal and linear(multilinear) and for areal objects, all points of one objectare checked for containing all points of other object.
At the first stage, when checking the inclusion rela-tionship between areal and linear (multilinear) objects,a check is performed for the nesting of rectangles inwhich the object data is entered. If the rectangles arenot inscribed, then no further verification is performed.
233
This check allows you to significantly save computing re-sources and execution time. Then, to verify the inclusionof point objects and areal, the algorithm for determiningwhether a point belongs to a polygon is executed, andfor other objects that have successfully passed the firstcheck, this algorithm is performed for each point of theobject. As an algorithm for determining whether a pointbelongs to a polygon, the ray tracing method is chosen.
After defining inclusion relationship between terrainobjects, they are formed in graph structures designed tostore the relationship (Fig. 4).
Figure 4. Formation in memory of the established relationship ofinclusion between the terrain objects ("Myadel district includes LakeNaroch")
The intersection relation is distinguished betweenlinear (multilinear) and polygon objects. The fact ofintersection of at least one part of one object and atleast one part of another object is checked. On the firststage rectangles are determined. If the rectangles have nointersections or inclusion, there is no further checking.Else procedure is being repeated for segment objects.If the fact of intersection is established, the objectsintersect, and then they are entered into formed graphstructures designed to store this type of relationship(Figure 5).
The "bordering" relationship stands out between arealobjects. The fact of coincidence of at least one part of oneobject and at least one part of another object of the terrainis checked. If there is a coincidence and the objects donot overlap, then a “border” relationship is establishedfor them, which is written into the graph structure (Fig.6).
The relation "contiguity" is allocated between linear(multilinear) objects. The fact of contiguity of the endsof an object with any part of another object is checked. Atthe first stage, the rectangles in which the scanned objectsare inscribed are determined. If the data rectangles do notintersect and do not fit into each other, further verificationis not performed. Otherwise, the procedure is repeatedfor the first object of each segment of the second. Ifthe rectangles that are inscribed inscribed object anda segment intersect or fit into one another, then thealgorithm checks the belonging from the ends of the first
Figure 5. Formation in memory of the established relationship ofintersection between terrain objects ("The Zachodniaja Dzvina Rivercrosses the city of Polack")
Figure 6. Formation in memory of the established relationship "bor-dering" between the objects of the area ("Belarus is bordered by theLithuanian")
object to this segment. In the event that the ownership isestablished, the objects are adjacent to each other.
If a membership established, the objects have a com-mon border to each other and the corresponding graphstructure is formed in the memory (Fig. 7).
When recording the junction relation for the rivers, thefact that the rivers consist of segments having differentattributes or their meanings is taken into account. On thebasis of this, an additional relationship is established forthe rivers, decomplexing. Each river is decomposed intosegments, and those in turn adjoin each other.
It should be noted that the formed topological-234
Figure 7. Formation in memory of the established relationship ofjunction between objects of the terrain ("The Svislach River contiguitythe Biarezina River")
geographical relations between the objects of the terrainmay have their own interpretations in the subject domain.For example, the formed relation "The Svislach Rivercontiguity the Biarezina River" in the knowledge base ofthe intellectual system will be interpreted as the "SvislachRiver adjoins into the Biarezina River".
B. Operations for semantic mapping of terrain objects
In addition to establishing topological relationships,for the correct and unambiguous storage of terrain objectsin the knowledge base of an intelligent system, it isnecessary to carry out a semantic comparison of geo-graphic objects. The semantic comparison of geographi-cal objects of the map occurs according to the followingprinciple:
– the object class is determined;– subclass, type, subspecies, etc. are determined. ob-
ject in accordance with the classification of terrainobjects, i.e. types of terrain objects in ontology;
– attributes and characteristics that are inherent in thisclass of terrain objects are determined;
– defines the values of the characteristics for this classof object;
– the homonymy of identification is eliminated;– corresponding links are established between the map
object, knowledge base object and object attributes;– establishes topological relationships between map
objects related to specific classes and types.As a result of these actions, the first version of the
knowledge base is formed, in which there are mapobjects with established topological relationships. As aresult, new knowledge is formed.
The second version of the knowledge base is the resultof the integration of the knowledge base, obtained at thefirst stage and the external subject-oriented knowledgebases. Such integration allows you to fill a knowledge
base with new types of knowledge, as well as to eliminatethe homonymy of geographic objects. For example, usingthe knowledge base of settlements with codes accordingto the SOATDS (system of designations of objects ofadministrative-territorial division and settlements) allowsidentifying settlements with the same name but belongingto different administrative-territorial units in a one-to-onemanner.
C. Integration of subject solvers
An advantage of OSTIS technology used is a pos-sibility of using problem solvers. Thus, such problemas implementing a search for routes between specificterrain objects, calculating geometric characteristics (forexample, an area of a territory) are performed by agentsthat can be implemented within the OSTIS technology,and also be third-party applications initiated to solveapplied domain problems.
On Figure 8 a specification of task for the subject do-main is shown. The result of processing with intellectualagents is shown on Figure 9.
Figure 8. Initial specification of an agent processing
Figure 9. Result of an agent processing
IV. INTEGRATION OF THE CARTOGRAPHICINTERFACE AS A NATURAL WAY FOR PEOPLE TO
REPRESENT INFORMATION ABOUT TERRAIN OBJECTS
The proposed principle of coding geo-informationallows the comparison of terrain objects described within
235
the framework of the subject domain and currentlyknown services of cartographic information in the In-ternet environment. Thus, visualization of terrain objectsstored in knowledge bases using Google, Yandex andOpen Map Street services is possible, as well as semanticcomparison of geographical objects with a knowledgebase with stored knowledge about these terrain objectsbased on the described principles in this article.
On Figure 10 a scheme of cartographic data visualiza-tion is shown.
Figure 10. Scheme of cartographic data visualization
V. EXAMPLE OF USAGE
One of the "advantages" of intelligent systems de-veloped using the OSTIS technology is the solution ofobjective problems when there is no clear specificationand algorithm to solve it. This is achieved by formingproducts that are recorded and stored also in the knowl-edge base Design technology of knowledge processingmachines and models for solving problems.
As an example, consider the solution to the followingproblem: "Determine whether there is a water routebetween the Min cities If there is such a waterway,display the result on the map". The following statementsof the knowledge base are the basic data:
1) The Svislach River flows through the city ofMinsk.
2) The Dnieper River flows through the city of Re-chitsa.
3) The Svislach River is a tributary of the BiarezinaRiver.
4) The Biarezina River is a tributary of the DnieperRiver.
During the first iteration, a "contiguity" will be es-tablished at the first iteration, analyzing the topologicalrelations:
1) The Svislach River flows into the Biarezina River(i.e. the Svislach River near the city of G.);
2) The Biarezina River flows into the Dnieper River(i.e. the Biarezina River adjoins);
And it will be concluded that the Svislach River can beused.
At the second iteration the city of Minsk and theSvislach River, as well as the city of Rechitsa and theDnieper River. As a result, the path that is displayed onthe map of the area will be found.
REFERENCES
[1] Xaun Shi “The Semantics of web Services: An Examination inGIScience Application,” ISCPRS Inetrnational Journal of Geo-Information, pp. 888-907, February 2015.
[2] Burcu Alinci and Hassan Karini “CAD and GIS Interoperabilitythrough Semantic web services,” Electronic Journal of Informa-tion Technology in Construction.
[3] A. Diostenu and L. Cotfas “Agent Based Knowledge ManagementSolution using Ontology, Semantic Web Services and GIS,”Informatics Economics, vol. 13, pp. 90–98, April 2004.
[4] Gregor J.Nalepa and T. Furmanska “Review of Semantic WebTechnologies for GIS,” Automatyka, vol.13, pp. 485–492.
[5] F. Fonseca, M. Egenhofer, P. Agouris and C. Camara “Us-ing Ontologies for Integrated Geographic Information Systems,”Transactions in GIS, vol.6, March 2002.
[6] Mahadi Farnaghi and Ali Mansourian “Multi-Agent Planning forAutomatic Geospatial Web Service Composition in Geoportals,”ISCPRS Inetrnational Journal of Geo-Information, July 2018.
[7] Din Le Dat and V. Serebryakov “Designing and Development ofFormal Ontologies of Geographical Spatial Data and Services,”Robototechnics Journal, vol. 2, pp. 85–89, 2008.
[8] V. Golenkov “Ontology-Based Design of Intellegent Systems,”Materials of Scientific and Technical International ConferenceOSTIS ,vol. 1, pp. 37–56, February 2017.
[9] “Digital maps of the area. Topographic information displayed ontopographic maps and city plans,” NCRB 012-2007.
[10] D. Shunkevich “Ontology-Based Design of Knowledge Process-ing Machines,” Materials of Scientific and Technical InternationalConference OSTIS ,vol. 1, pp. 73–94, February 2017.
[11] V.B. Taranchuk “New computer technologies, analysis and in-terpretation of geodata,” MATEC Web of Conferences IPICSE-2018. V. 251, 04059. VI International Scientific Conference"Integration, Partnership and Innovation in Construction Scienceand Education" (IPICSE-2018). – P. 1-8.
СЕМАНТИЧЕСКАЯ ТЕХНОЛОГИЯПРОЕКТИРОВАНИЯ ИНТЕЛЛЕКТУАЛЬНЫХ
ГЕОИНФОРМАЦИОННЫХ СИСТЕМ
Самодумкин С.А.
Настоящая работа посвящена вопросам созданиячастной технологии проектирования интеллектуаль-ных геоинформационых систем, использующих знанияоб объектах местности для решения прикладных задачв проблемных областях.
Received 10.01.19
236
Analysis of Semantic Probabilistic InferenceControl Method in Multiagent Foraging Task
Vitaly Vorobiev, Maksim RovboNational Research Center «Kurchatov Institute»
Moscow, 123182, [email protected], [email protected]
Abstract—Adaptation in robotics systems is often imple-mented as some form of learning. While much research isdedicated to studying policy and value approximation inreinforcement learning, some methods are based on ruleinference and logical descriptions. One of these methodsis based on a semantic probabilistic inference algorithmthat has its roots in the theory of functional systems.In this article, the method is applied to a distributedmultiagent foraging problem that has an important prop-erty of providing an environment that allows to study adecentralized system of individually learning agents. Wecompare the performance of this method to other methods:Q-learning and a random choice algorithm as a baseline.We also propose a modification of the algorithm thatincludes an exploration behavior. Experiments are carriedout in a computer simulation system. The results show theperformance of the algorithms with different parameters,as well as the effect of exploration on the performance.
Keywords—adaptive control, robotics, semantic proba-bilistic inference, foraging, local interaction
I. INTRODUCTION
Adaptive control system for robotics are of practicalinterest since they promise to increase robustness ofexisting systems, make the behavior closer to optimal aswell as introduce the possibility to impart new behaviorsto the robot by a system of rewards or examples. Thismay be especially important for multiagent systems ascontrolling them in a direct way to achieve a given goalis harder than single robots.
A lot of current research is dedicated to learningmethods for virtual and robotic agents that is based onreinforcement learning methods using value and policyapproximations, especially based on parametric descrip-tions of the functions and neural networks. Multiagentaspect introduces event more problems, like operating ina dynamic, non-markovian environment that makes evena static environment more challenging due to the activityof the agents themselves in relation to other agents.Robots often work in an environment, where only someinformation about the state is accessible, which meansmaking decisions in a partially observable environment.Thus, seeking efficient ways to search the policy spacefor acceptable (and, preferably, optimal) observation-action mappings is important.
One of the ways to address this problem it to seekbiologically inspired models of decision making or using
different representations of the problem and policy space,such as logical. There are various approaches and meth-ods that use logical descriptions that could be used fordecision making, for example, semiotic networks [10],JSM method [2], semantic probabilistic inference [7], [9].
Semantic probabilistic inference (SPI) is a learningmethod for an agent that uses logical (rule-based) de-scriptions of the actions of the agent that was introducedin [9]. It is based on a mathematically formalized conceptof a functional system from the theory of functionalsystems [7].
One of the main goals of this work was to studyand compare capabilities of the SPI method and somereinforcement learning methods in a multiagent settingwith physically distributed agents and also the effects ofexploration and some other, problem-specific parameters,on the agents’ performance. While SPI was used asa basis of a network composed of logic neurons andstudied in a multiagent context in [1], in those worksagents controlled tightly coupled (physically connected)elements of a robot, used a common reward from acentralized source and inferred the rules in a singlesystem that could create rules specific to each agent, aswell as general rules. In this work we emphasize that thestudied system does not provide agents with a commonreward (each reward is specific to the agent), the agentshave only an indirect effect on each other’s performanceand they do not have to use a centralized rule-basedlearning system, but can learn separately from each other.
The chosen problem environment is a foraging prob-lem, where agents must gather food units in a gridworld since it can be seen as a reasonably representativeproblem for some simple group robotics tasks and itsatisfies the environment requirements described above.We also propose a modification of the SPI algorithmthat introduces exploration behavior into the system sothat the agent is less susceptible to local minima ofperformance, especially in a stochastic environment.
On the other hand, an important issue is the question ofthe application of the logical system of adaptive control,which is based on the algorithm of semantic probabilisticinference, to a group of mobile robots that allows localinteraction. In this regard, it is proposed to consider the
237
possibility of using such a control system for a groupof robots that solve some common task. In this case,the main emphasis is placed not on the solution of thecommon task but on resolving the problem of organizingcommunication and capabilities of separated and movingrobots.
The work is structured as follows. Firstly, methodsand algorithms used in the paper are presented, as wellas the proposed modification. Then the model of theproblem is described. After that, the experiment param-eters, simulation results and the analysis follow. Then,in the section “Organization of a group of robots forcollective application of the logical model of an adaptivecontrol system” further research is described, detailingthe problems that need to be solved and a proposedapproach to adapt the semantic probabilistic inference fora physically distributed group of mobile robots. Finally,a conclusion sums up some key points about the article.
II. METHODS AND ALGORITHMS
The main algorithm that is studied in this work is thesemantic probabilistic inference that is described in [9]but without the mechanism of new functional systemformation as it was observed in the original work thatfor a foraging problem forming new functional systemsis not required. The algorithm also uses the concept ofa goal predicate, but in the later works [1] a rewardwas used as a prediction, which is what we use here,but without creating logic neurons described in the latterwork. The reward predicted by the rules always equalsto one, so it can be written as a goal predicate that statesthat the agent gets a reward of one.
We also propose a modification of the algorithm byintroducing an exploration behavior into the system.The original algorithm chooses a random action onlyin cases where the situation was never encounteredbefore and / or there were no suitable rules inferredfrom the experience. Instead, we also add a randompossibility of choosing an action randomly with uniformprobability that is governed by an exploration rate ε.This is similar to exploration done by the Q-learningalgorithm and should help the agent gather informationabout alternative choices of actions in situations thatalready have a suitable rule. There are also cases wheresuch exploration strategies were successfully applied toforaging problems [6], so it seems reasonable to try itfor the SPI algorithm in a multiagent setting.
The following is a short version of the SPI algorithmas it is implemented in this work. The proposed modifica-tion is marked by an asterisk and is basically everythingthat uses ε.
1) Parameters of the algorithm basic_rule_depth brd,max_plan_length mpl are set, the environment andagents are initialized.
2) Agent receives an observation obs from the environment,which includes the reward r from the previous action.
3) Agent updates its experience table (called here spi_table)by adding 1 to the record describing a combination of thelast state slast, sequence of last actions taken aseqlastand the resulting reward r (0 or 1):
spi_table[slast, aseqlast, r] += 1
4) Rules for regularities detection are created by exploringa graph that has rule of the following form as its nodes:
P1 ∧ P2 ∧ ... ∧A1 ∧ ... ∧Ampl → r
which contain state predicates (P1 can be, for example,a fact “the left cell has food”) and a sequence of actionsAi, in the precondition and a predicted reward r in thepostcondition that always equals 1 (otherwise the rulewould never be applied). Nodes are explored in twosteps. Firstly, all possible rules with no more than brdpredicates in the precondition including action predicatesand no less than one action are built by expanding a nodewith a single new predicated added to the preconditions.During the second stage, only the rules that pass apositive rule regularity check are expanded. The firstnode is a rule→ r.
5) Positive regularity check for a rule means that its es-timated probability to yield a reward r is higher thanthat of any subrule that can be formed with a subsetof its preconditions. Only rules that pass a positiveregularity check are added to the list of regularities fordecision making. The probability check is the followinginequality:
n(Prule ∧Arule ∧ r)
n(Prule ∧Arule)>
n(Psubrule ∧Asubrule ∧ r)
n(Psubrule ∧Asubrule)
where Prule — state preconditions of the rule, Arule
— action preconditions (planned actions) of the rule,Psubrule — state preconditions of the subset rule,Asubrule — action preconditions (planned actions) ofthe subset rule, n(predicates) — number of times thepredicates were applicable to agent’s situation stored inits experience table spi_table.
6) (*) Exploration action is carried out with ε probability,which is a randomly chosen action with a uniformprobability and step 9 is performed. Otherwise the usualSPI action selection is applied.
7) If exploration action was not chosen, all discoveredrules’ applicability to current state from the regularitieslist are checked. If a rule’s precondition is satisfied,its performance rule_performance (probability to geta reward following the precondition actions from thecurrent state) is checked according to the formula:
rule_performance(rule, state) =
=p(Pstate ∧Arule ∧ r)
p(Pstate ∧Arule)
where Pstate is all predicated describing the currentobservation (not just the predicated from the rule’sprecondition). A list of such rules is formed with theirperformances.
8) The applicable rule with the highest performance ischosen and its first action (if there are several in therule action plan) is chosen to be performed:
chosen_rule =
= argmaxrule
(rule_performance(rule, state))
238
action = (Achosen_rule)1
9) The chosen action is stored as the last action performed,the sequence of length mpl of last actions is updated,the current state is remembered as the last state and theaction is returned to the environment to be performed.
The SPI method was expected to show relatively highperformance, surpassing some classical reinforcementlearning algorithms at least on initial episodes, sinceit can aggregate states from observation space by onlydeciding on a few variables (predicates) from it. Theoriginal (with exploration rate ε = 0) SPI also quicklyadapts rewarding behavior, but it can be hypothesizedthat it can fall into a local minima because it doesnot seek new rules actively for a state that already hasa rewarding regularity discovered. Hence the proposalto add an exploration coefficient to it, so that it cansometimes check other actions.
A random choice algorithm was used as a baseline forcomparison. Another algorithm to compare the perfor-mance against was the classical Q-learning (in tabularform), described, for example, in [4]. It maps states toactions by using a value function Q(s, a) that serves asan estimate of how much reward an agent can get froma given state s by choosing an action a. The actions arechosen to maximize the reward:
a(s) = argmaxa
(Q(s, a))
The state-action values are stored in a table and areupdated using both the actual received reward r at thecurrent step after receiving information about the states’ the agent was transferred to and the estimate of thenext rewards:
Q(s, a) = Q(s, a) + α(r + γ ·maxaQ(s′, a)−Q(s, a))
The estimate is updated with a learning rate α that canbe interpreted as indication of how much the observationsare trusted and the discount rate γ, which indicates howmuch the agent values (or trusts the estimates of) thefuture rewards.
It should be noted that in partially observable en-vironments observations must be used instead of thewhole states and that the method uses the whole stateinformation to make decisions. It means that if only onesmall part of the observation changes, the agent treatsthe situation as a completely new one and will have tolearn it from scratch.
III. EXPERIMENTAL SETUP
Foraging problem is a relatively common testing envi-ronment for multiagent reinforcement learning problems,but can be formulated differently. Here it is defined in thefollowing way. There are three types of objects – foodunits, obstacles and agents situated in square grid cells.Neither agents nor food can be located on a cell withan obstacle, but there can be unlimited amount of food
units or agents on any of the free cells. The obstaclesare located only at the edges of the field (Fig. 1):
Figure 1. Foraging environment. Green circles are food locations, pinkcircle is an agent and the white line on it shows the direction it iscurrently facing, light squares on the fringes are obstacles.
Agent can move and when moving onto a cell withfood objects it «eats» one of them and gets a reward of1 as part of the observation on its next step. This is tohave a unified and more realistic interface between theagent and simulation, since rewards are not a separateentity of the world, but rather an interpretation of theagent of the situation or an external signal that explicitlycommunicates a reward. Agent has a direction, where itis currently facing. It can choose one of three actions:moving forward one step in the direction it is facing,turning 90 degrees left or right.
An observation consists of a number of nearby cellsand their simplified contents. The simplification is thatonly a type of the object is shown on the grid to theagent and none of its internal parameters, and whenthere are multiple objects in a cell, only one of themis detected. The exact cells to be observable dependon the observation radius and are calculated using amaximum norm. The direction of the agent is facing alsodetermines the observation so that the agent also faces‘up’. For example, for radius of one the observation looksas shown in Fig. 2:
Figure 2. Observation with vision radius equal to 1. In this example,three obstacles are shown to the right, two unites of food are locatedbehind the agent with one being diagonally to the left, the agent itselfis in the center and it always looks forward (‘up’) in its local systemof observation, and the reward received on the previous step is 0.
The goal of each agent is to maximize the cumulativereward (food it gathered) over an episode, where anepisode is a fixed length sequence of steps. For con-
239
venience, instead of a rolling sum, all of the steps areseparated into non-overlapping episodes.
IV. RESULTS AND DISCUSSION
The model of the world consisted of a 25 by 25 gridwith 100 randomly placed food units. The amount offood on the field was constant which was provided byrandomly placing another unit of food whenever a unitwas gathered by an agent. There were simultaneously 10agents on the grid with the same starting parameters, butthat evolved independently of each other (without delib-erate communication). At the start of each step, all agentsdecided on an action based on the observation of nearbysurroundings with a vision radius of 1 (all contiguous toan agent cells, including diagonal, were seen by it). Thenthe simulation executed each of the actions sequentially.On the next step, agents were provided with a reward (ifthey gathered a unit of food on the previous step) of 1by including that information in the observation.
Experiments were carried out 10 times for each setof parameters and the results were averaged over theexperiments. They were further averaged over the agentsto get a learning curve for an algorithm with a set ofparameters. While a standard practice for single agentsimulations in reinforcement learning, it also makessense in this case for a multiagent problem, since agentsof the same type (with the same algorithm and startingparameters) show similar average performance.
When a new experiment starts, the world and agentsreset (learned parameters are not kept between exper-iments). Each experiment consisted of 2000 steps thatare grouped into episodes — each episode is a 100steps. The episodes are mostly a more convenient way toview the results of a simulation, since the task itself iscontinuous. Each episode’s cumulative reward for eachagent is recorded.
Semantic probabilistic inference had its rule depthlimited to 1 (rules with more than one precondition wereretained only after a successful regularity check wasperformed) and its plan length to 1 (so that no morethan one action is in a precondition). It was tested bothwith the proposed exploration modification and severalexploration rates ε., as well as without it as in the originalarticle [9], which is equivalent to ε = 0.
Q-learning parameters used in the experiments are thefollowing: exploration rate ε = 0.05, learning rate α =0.1, discount factor γ = 0.1.
Computational experiments were carried out in a cus-tom Python simulation system. Experimental results areshown in Fig. 3.
The random learning agent serves as a baseline forcomparison and also allows to check that the environmentis not too easy for an agent — rewards must be relativelylow between a random and a learning agent.
The foraging problem itself in this formulation hasseveral interesting qualities — locally (within a single
Figure 3. SPI – semantic probabilistic inference algorithm. Thenumber after ’SPI’ is the ε used. Random algorithm is a baseline forcomparison. The bold line is the mean over all agents and experiments,the shaded area shows minimum and maximum performance betweenagents with the same algorithm.
agent’s viewfield) it is almost deterministic, that is,correct actions will always lead to the same reward,unless another agent interferes, which is relatively rarefor such rate of agents to the field size (less than 2%of cells are occupied by agents, which observe at mostabout 14% of the world). Outside of the immediate ob-servation area, however, the world is highly unpredictableto agents, since they only have a partial observationof the world state (and a considerably limited, at that),with food spawning randomly and agents having almostno information, which could allow to determine foodlocation outside of the immediate observation.
This leads to a behavior that is similar to randomchoice when no food is observed (although, agents showa preference for forward movement after learning) anda set of rules that instruct to pick up observed food assoon as it appears in the agent’s view. However, this en-vironment did not demonstrate significant opportunitiesto trap an agent’s policy in a local minimum because ofthe aforementioned properties. Most random significantdrops in an agent’s performance can be attributed tohaving collected all the food in a local area and randomlymoving in a now-empty area that the agent has no wayof determining without sufficient memory.
This might contribute to lower importance of explo-ration in this problem, despite it usually being importantto get a high performance for some types of learningagents, as seen from, for example, comparison in thework [5]. We can also observe that high values (0.5 onthe graph) lead to a decreased performance, which isexpected since even optimal rules would be ignored halfof the time.
Q-learning quickly becomes overwhelmed by the statespace when increasing the radius of the agent vision andlearns very slowly, while showing performance below
240
that of a random agent. It is not shown on the graph, butseparate experiments have shown that with vision radiusequal to 2 cells away, Q-learning surpasses performanceof 50 by about 200th episode (which is 200 000 steps).
V. ORGANIZATION OF A GROUP OF ROBOTS FORCOLLECTIVE APPLICATION OF THE LOGICAL MODEL
OF AN ADAPTIVE CONTROL SYSTEM
Firstly, let’s consider, instead of a shortened versionstudied above, a full logical model of an adaptive controlsystem that is introduced in [9]. It can be formally writtenas CS = 〈S,A, P, F 〉, where F is the description of thehierarchy of the basic elements of the system, which arefunctional systems, as S = S1, S2, ..., Sn is the setof sensors of the robot (a robot with such structure willbe further called animat), A = A1, A2, ..., Am is theset of its possible actions and P = P1, P2, ..., Pl —the sensory information at a specific time in the form ofpredicates which can describe not only the current, butalso the past states of the sensors.
Each functional system (FS) is a tuple FS =〈PG,G, PR〉, where PG is a predicate-goal, describesa goal that is represented using the conjunction ofsensory predicates P, i.e. PG = P1 ∧ P2 ∧ ... ∧ Pl. Ifthis predicate is true, then the goal is achieved. G =PG1, PG2, ..., PGn, where PGi are goal predicatescorresponding to the goals of the subordinate FS in thehierarchy. PR is a pattern, in the form of:
P1∧P2∧...∧Pn∧PG1∧PG2∧...∧PGm∧A1∧A2∧...∧Ak
which shows that:• If the animat is in the state described by the sensory
predicates P1, P2, ..., Pn;• If in this situation it sequentially reaches the sub-
goals PG1, PG2, ..., PGm;• If then it sequentially performs actionsA1, A2, ..., Ak;
• Then it will reach the goal G with some probability.With a given goal or sub-goal and known information
about the world and the internal state of the FS, the taskof this FS is to find the best way to achieve the goalby performing the actions chosen on the basis of theprediction.
The example of general scheme of this architecture isshown in Fig. 4
Figure 4. Example of adaptive control system architecture.
Since this case describes the model of the adaptivecontrol system for a single animat, then considering a
group of robots with local interaction, each robot in thegroup can be associated with a separate FS. The linksbetween individual FSs are then identical with the locallinks between the robots of the group. Therefore, orga-nizing the processes of transferring subgoals from higher-level FS-robots to lower-level FS-robots and transferringresults or predictions of results from lower-level FS-robots, you can ensure that the algorithm for semanticprobabilistic inference for a group of robots is similarto the same algorithm for a single robot. This allows totreat a group of robots as a single goal-directed entitydriven by the SPI algorithm.
Thus, to adapt the logical model of the adaptive controlsystem for a group of robots, it is necessary to builda hierarchy of relations within the group of robots,each of which is known for the adaptive control systemarchitecture.
This first problem can be solved in two stages: first,a leader is selected in the group of robots that will bethe top of the hierarchy. Restriction on the locality ofinteractions between agents makes leader selection non-trivial, but algorithms that can solve this problem aredescribed in, for example, [3] or [8].
By applying one of these algorithms [8], which isbased on the redistribution of weights between robots,a hierarchical communication structure can be created(Fig. 5):
Figure 5. Hierarchical structure resulting from the use of a leaderselection algorithm for a group of robots with local interaction.
It can be seen from the figure that it is a structuresimilar to the structure necessary for the functioning ofthe logical model of the adaptive control system, wherethe root agent ‘0’ corresponds to the one of FS.
However, there are still links connecting several robotsat the top level with the same robot at a lower level. Inthis regard, the second stage of preparation is necessary,i.e. carrying out the procedure for removing such con-nections, which consists in sending the leader a specialmessage M with his number to his neighbors. As soonas the message M is received, the robot remembers thenumber of the “parent” robot from which it received itand will continue to receive messages only from it. Thenhe sends the message M with his number and the number
241
of his “parent” to his neighbors. Neighbors of the upperlevel, except for the “parent”, by accepting this message,will exclude him from their neighbors. Lower levelneighbors will remember him as their “parent” unlessthey already have another. If messages from different“parents” came at the same time, a “parent” with a largenumber is selected.
This is a particular example of when the connectionsin the adaptive control system form a tree. In general,there is no need to remove additional links.
It remains to assign the remaining FS robots from F (k)
in such a way that they satisfy the expressions describingthe set F (k). This is achieved by assigning each neighborof the robot a separate branch of a common tree, whichdescribes the scheme of relations of individual FS witheach other in the adaptive control system. Each neighborthus gets its own tree for distributing its branches amongits neighbors, etc. Obviously, with such a physical orga-nization, a number of restrictions appear, for example,the number of lower-level FS that can be controlled bythe upper-level FS is limited by the maximum numberof local communication channels of the robot.
Consequently, in a given group of robots with localinteraction, only some functional system hierarchies canbe implemented. Another factor limiting the numberof possible hierarchies is the number of robots in thegroup, i.e. it is impossible to implement systems with|F (k)| > R, where R is the number of robots in a group.Moreover, even if |F (k)| > R, it does not guaranteethat robots will be able to organize such a controlsystem. This is due to the fact that it is impossible topredict what hierarchical structure robots will build in theprocess of self-organization, knowing only their numbersand the number of their possible local neighbors. Theprobability of building a suitable structure increases ifR is noticeably greater than |F (k)| and increasing L.In other words, the more robots in a group, and themore neighbors each robot can have, the more complexfunctional system hierarchy they can reproduce.
A similar approach can be used to create a group ofrobots with local interaction, which can be controlled asa whole using a logical model of an adaptive controlsystem.
ACKNOWLEDGMENT
This work was supported in part by grant RFBR 18-37-00498 mol_a (parts concerned with the semantic prob-abilistic inference) and RFBR 17-29-07083 (multiagentQ-learning).
REFERENCES
[1] Demin A.V., Vityaev E.E. Adaptive Control of Modular Robots.Biologically Inspired Cognitive Architectures (BICA) for YoungScientists. BICA 2017. Advances in Intelligent Systems andComputing, 2018, vol. 636, pp. 204-212.
[2] Finn V.K. Plausible inferences and plausible reasoning. Journalof Soviet Mathematics, 1991, vol. 56, no 1, pp. 2201-2248.
[3] Karpov V., Karpova I. Leader election algorithms for staticswarms. Biologically Inspired Cognitive Architectures, 2015, vol.12, pp. 54-64.
[4] Sutton R.S., Barto A.G. Reinforcement Learning: An Introduc-tion, Cambridge, MA: The MIT Press, 2018, 552 p.
[5] Tokic M. Adaptive ε-greedy exploration in reinforcement learningbased on value differences. Lecture Notes in Computer Science(Including Subseries Lecture Notes in Artificial Intelligence andLecture Notes in Bioinformatics), 2010, vol. 6359 LNAI, pp. 203-210.
[6] Yogeswaran M., Ponnambalam S.G. Reinforcement learning:Exploration-exploitation dilemma in multi-agent foraging task.Opsearch, 2012, vol. 49, no 3, pp. 223-236.
[7] Vityaev E.E. Printsipy raboty mozga, soderzhashchiesya v teoriifunktsional’nykh sistem P.K. Anokhina i teorii emotsii P.V. Si-monova [The principles of the brain from the Anokhin’s theoryof functional systems and P.V. Simov’s theory of emotions].Neiroinformatika [Neuroinformatics], 2008, vol. 3, no 1, pp. 25-78.
[8] Vorobiev V.V. Algoritmy vybora lidera i klasterizatsii v statich-eskom roe robotov [Leader choice and clustering algorithmsin a static swarm of robots]. Mekhatronika, avtomatizatsiya,Upravlenie [Mechatronics, automation, control], 2017, vol. 18,no 3., pp. 166-172.
[9] Demin A.V., Vityaev E.E. Logicheskaya model’ adaptivnoi sis-temy upravleniya [Logical model of the adaptive control system].Neiroinformatika [Neuroinformatics], 2008, vol. 3, no 1, pp. 79-107.
[10] Osipov G.S., et al. Znakovaya kartina mira sub"ekta povedeniya[Symbolic worldview of a subject of behavior]. Moscow, FIZ-MATLIT, 2017, 259 p.
АНАЛИЗ МЕТОДА УПРАВЛЕНИЯ НА ОСНОВЕСЕМАНТИЧЕСКОГО ВЕРОЯТНОСТНОГОВЫВОДА В МНОГОАГЕНТНОЙ ЗАДАЧЕ
ФУРАЖИРОВКИ
Воробьев В. В., Ровбо М. А.
Адаптация в робототехнических системах частопредставляет собой какую-либо форму обучения. Хо-тя многие исследования посвящены изучению прибли-жения стратегии и функции полезности в обучении сподкреплением, некоторые методы основываются навыводе правил и логическом описании. Один из нихоснован на алгоритме семантического вероятностноговывода, который имеет корни в теории функцио-нальных систем. В этой статье метод применяетсяк распределенной многоагентной проблеме фуражи-ровки, которая имеет важное свойство в виде среды,позволяющей изучать децентрализованную системуиндивидуально обучающихся агентов. Мы сравниваемэффективность этого метода с другими: Q-обученияи алгоритма случайного выбора в качестве основысравнения. Мы также предлагаем модификацию ал-горитма, включающую исследовательское поведение.Эксперименты проведвены в системе компьютерно-го моделирования. Результаты показывают эффектив-ность работы алгоритмов для различных параметров,а также влияние исследовательского поведения.
Received 10.01.19
242
On Ontological Modeling of Measurements ina Complex Monitoring System
of Technical ObjectMaria N. Koroleva
Bauman Moscow State Technical UniversityMoscow, Russia
Georgy B. BurdoTver State Technical University
Tver, [email protected]
Abstract—A complex problem of monitoring sophisti-cated technical object (bridge) is faced. Some importantpeculiarities of measurement tasks and their organization inmonitoring system are described. In order to represent andengineer measurement knowledge an ontological approachto measurement specification is presented. A hierarchicalsystem of measurement ontologies is proposed, some basiclow-level ontologies are constructed by using mind maps.A special attention is paid to the analysis of uncertaintytypes in measurement.
Keywords—ontology; measurement; ontological model-ing; ontological hierarchy; measurement uncertainty; sen-sor networks; monitoring
I. INTRODUCTION
The new industrial revolution under the name of Industry4.0 based on Cyber-Physical Systems and Internet of Thingssupposes the concept formation and implementation of so-called Ubiquitous Measurements with Sensor Networks. Suchnetworks can be viewed as a community of autonomous agentslocated in different places and maintaining communicationsto generate a distributed cognition system. Enabling mutualunderstanding and joint work of these agents requires a systemof measurement ontologies. This problem is faced in our paperin the context of monitoring sophisticated objects in railwayinfrastructure (by taking an example of bridge).
II. MEASUREMENT IN A COMPLEX MONITORINGSYSTEM
A complex problem of bridge monitoring (Fig. 1) includesthe following tasks:
1) specification of keynote characteristics of the bridge state(e.g. bridge deformation, uneven draft of the structure,vibrations), measurement of main meteorological param-eters (first of all, wind strength and direction, etc.);
2) interpretation of measurement results;3) analysis of the processes in the construction of the bridge
and diagnostics of its current state;4) prognosis of the further evolution in the state of the
bridge structures;5) decision making related to possibility and safety of
bridge operation.Thus, measurements are a principal information source toperform subsequent monitoring tasks.
This work is supported by RFBR, grant No 18-07-01311 and 19-07-01208
Figure 1. Tasks to be performed in the course of monitoring
According to the branch road methodical document [1]monitoring means an experimental checking of quantitativeparameters (measurement) and qualitative factors specifying atechnical state of the bridge. These are:
• geometrical parameters;• stress-strain state;• temperature of bridge structures;• dynamic characteristics;• defects;• loads and impacts;• atmospheric conditions of bridge operation;• stiffness, strength and other properties of structures and
materials.Both current parameters values and their changes while mon-itoring can be specified. Measurements in monitoring can beperformed by using both devices with continuous data registra-tion and in the form of periodic instrumental measurements byusing sensors and devices pre-installed in the bridge structures.
Measurement theory encompasses knowledge about mea-surement types, methods, tools, instruments, results, conditions.In order to develop metrological intelligent systems on the basisof knowledge engineering [2] let us consider an ontologicalapproach to measurement.
III. ONTOLOGICAL APPROACH TO SPECIFYINGMEASUREMENT
Ontology (in Computer Science) is usually seen as a for-malized description of some problem area. Two classical ap-proaches in ontological modeling are known —- relational andlogical; these approaches rise to the foundational papers byT.Gruber [3] and N.Guarino [4]. So Gruber defines ontologyas an explicit formal specification of a conceptualization sharedby the members of some community. Here basic keywords are:
243
• ”conceptualization” – synthesis of abstract conceptualmodel of external world phenomena by identifying key-stone concepts and studying main relations between them;
• ”formal” – such a conceptualization should be expressedin a machine readable format to be understood by com-puter (sensor) system;
• ”explicit” – it means that the type of concepts used andthe constraints of their use are explicitly defined;
• ”shared” – ontology captures consensual knowledge ac-cepted by a group in order to enable mutual understandingand joint work of various agents (a communicative aspectof ontology).
According to Guarino, ontology is a logical theory thatgives an explicit partial account of a conceptualization. Itincludes some basic terms forming taxonomy, their definitionsand attributes, related axioms and inference rules.
It is often very difficult or even impossible to construct sin-gle, comprehensive, coherent and practically useful ontology.To simplify ontology development and reuse, a modular ap-proach is taken and some hierarchies of ontologies are formed[4]–[6]. On the low level apart from domain ontology, bothtask ontology and application ontology are constructed, andon the high level upper ontologies [7] are viewed to representgeneral categories encountered in many problem areas.Besides,meta-ontology (”ontology of ontologies”) is given that providesboth an exact mathematical specification of various ontologiesand formal analysis of their properties. Specifically, it includesmethods and forms of representing, developing and mergingdifferent ontologies.
An example of foundational measurement ontology is QUDT(Quantity, Unit, Dimension, Type) ontology [8], where classproperties and restrictions are defined to model physical quan-tities, units of measure and their dimensions in various mea-surement systems. The goal of the QUDT ontology is toprovide a unified model of measurable quantities, units formeasuring different kinds of quantities, the numerical values ofquantities in different units of measure and the data structuresand data types used to store and manipulate these objects insoftware. Among other perspective ontological approaches tomeasurements Kuhn’s functional ontology of observation andmeasurement [9] should be mentioned.
Below a three-leveled system of measurement ontologies issuggested (Fig. 2) in the framework of solving complex moni-toring problem. Here low level ontologies include measurementdomain ontology, ontology of measurement properties, ontol-ogy of sensor networks as measurement tools, measurementapplications ontology (measurement for monitoring). Followingthe Guide to the Expression of Uncertainty in Measurement(GUM) (see [10], [11]), we take uncertainty ontology as upperontology. Finally, we have granular ontology [12] on the meta-level.
Such visual tools as mind maps [13] are worth employingto represent ontologies. A basic idea of mind map is toautomatically transform some text fragments in a graphicalform. Such a map possesses the following features:
1) it has a form of a bush;2) a studied object is placed in the center of the picture that
corresponds to the center of attention;3) primary topics related to the object of investigation
diverge from the center as branches explained withkeywords;
4) secondary topics are also branching;5) the branches form a connected nodal structure.Four low-level ontologies are depicted as mind maps in
Fig. 3 – Fig. 6: ontology of sensor networks, ontology ofmeasurements, ontology of measured properties (with using
Figure 2. Ontological hierarchy of measurements
Doynikov’s classification of measured properties [13]) andontology of bridge monitoring as application ontology formonitoring [15].
Figure 3. Visual representation of sensor network ontology
Figure 4. Mind map for measurements ontology
It is seen from Fig. 3 that the problem area ”SensorNetworks” is revealed through such concepts —- classes as”Sensors”, ”Networks”, ”Environment”, ”Application Goal”.Here measurement tasks ontology is tightly connected withmeasurement applications in the framework of monitoringproblem. Main measurement tasks in monitoring situation areboth formation of judgments and support of reasoning. Thesejudgments and reasoning concern diagnostics of current state ofmonitoring object (bridge), prognosis of their change tenden-cies, decision-making and recommendation development. Forexample, ”if the wind speed measured by the anemometer onthe bridge is 25-26 m/s and considerable bridge vibrations areobserved, then the traffic on the bridge is prohibited”. It is thecase of joint, multiple, dynamic measurements.
244
Figure 5. Mind map for ontology of measured properties
Figure 6. Mind map for ontology of bridge monitoring
IV. ON UNCERTAINTY TYPES IN MEASUREMENTS
Any measurement is an experiment under uncertainty.Thus it always has an error. Measurement results dependon measurement-information system, measurement techniques,external conditions, human-operator’s qualification and so on.An old idea of measurement with exact result largely hasoutlived itself.
In International and Russian standards an ordinary term”measurement error” or its inverse ”accuracy of measurement”was replaced by a wider term ”measurement uncertainty” [10],[11]. Measurement uncertainty is a general concept associatedwith each measurement. By taking into account measurementuncertainty, we are able to compare measurement results withexisting standards and norms, perform diagnosis of the monitor-ing object and prognosis of its future behavior, make importantpractical decisions and manage risks.
In [16] some analogies between classical measurement sci-ence and new concept of measurement uncertainty have beentraced. Nevertheless, the matter is not only terminologicaldifferences, but a quite new representation of the sense by mea-surement results. According to GUM, certain measurements donot exist.
In [10], [11] measurement uncertainty is viewed as a princi-pal lack of exact knowledge about measured value. In otherwords, the result of measuring some quantity x has twocomponents:
• some value x0;• its uncertainty ux.
Generally we have (x0±ux) mu, where mu means measure-ment unit.
The result of measurement is only an approximation ofmeasured value. It cannot be expressed by a singleton and it ischaracterized by a distribution on confidence interval.
A. Uncertainty of type A and BOn the one hand, in analyzing measurement uncertainty the
GUM materials [10] have to be taken into consideration. Onthe other hand, the limitations of classical stochastic techniquesrequire the development of more general approach to the mod-eling of uncertainty in measurement related to such concepts asgranule and measurement information granulation [17]–[19].
In [10], [11] the difference is made between type A andtype B uncertainty. Type A uncertainty evaluation is performedby the statistical analysis of series of observations. Type Buncertainty requires a new evaluation method other than thestatistical analysis of series of observations, for instance, fuzzyvariable, fuzzy interval, fuzzy number, possibility distribution.Here, type B uncertainty (more exactly, a complex of varioustypes of non-stochastic uncertainty) encompasses such factorsas:
• incomplete or inaccurate definition of the measure and,for instance, the lack of justified uncertainty value ux;
• a non-representative sample;• an open character and dynamics of measurement proce-
dures, their dependence on measurement goals, environ-ment and available instrumentation;
• instrumentation imperfection due to its finite resolution ordiscrimination threshold;
• inaccurate or incomplete knowledge of both environmen-tal conditions and their impact to measurement results.
B. Classification of type B uncertaintiesLet us take as a basis Borisov’s uncertainty classification [20]
that was introduced in 1980’s. In this classification uncertaintywas also divided into two classes – stochastic uncertainty andnon-stochastic (linguistic) uncertainty. It is suitable to modify itfor measurement with using sensor networks by taking into ac-count inaccuracy, incompleteness, ambiguity, contradictoriness,fuzziness in measurement (Fig. 7).
Figure 7. Classification of type B measurement uncertainties
Measurement inaccuracy means a limited sensor resolutioncapacity due to the nature of measured parameter.In everymeasurement there exists an unrecoverable error depending onthe sensor threshold. For instance, an acoustic range finder
245
determines the distance to objects within the limit of 0,2—80m with an error 2%.
Incomplete measurement information supposes that in sensornetwork a few sensors are inaccessible, and total ignorancemeans a loss of communication with the sensor or malfunctionof measuring element.
Contradictory indications may appear for homogeneous sen-sors measuring the same parameter. For example, two straingauges measure structural stress: the first sensor indication isinterpreted as ”normal”, and the second sensor indication isseen as ”out of norm”.
Measured data ambiguity supposes the use of some (non-probabilistic) distribution. Fuzzy value is attributed to the termsof linguistic variable. For instance, ”the measured wind speedon the bridge is almost in the norm”.
V. CONCLUSION
As a result of this research, some proposals on granularmeasurement uncertainty models and cognitive structures ofmeasurements have been introduced. In order to develop dis-tributed cognition systems a hierarchical system of ontologicalmeasurements has been proposed, basic low-level ontologiesare constructed. All above allows to develop intelligent moni-toring systems based on distributed cognition systems.
REFERENCES
[1] Handbook on Monitoring the State of Operated Bridges. TheBranch Road Methodical Document. Moscow: Avtodor Editions,2008 (in Russian).
[2] T. A. Gavrilova, D. V. Kudryavtsev and D. I. Muromtsev,Knowledge Engineering: Methods and Models. The Textbook.St.Petersburg: Lan’ Publishers, 2016 (in Russian).
[3] T. R. Gruber, ”A Translation Approach to Portable Ontologies”in Knowledge Acquisition, vol.5, #2, 1993, pp. 199–220.
[4] N. Guarino, ”Formal Ontology and Information Systems” inProceedings of the 1st International Conference on Formal On-tologies in Information Systems (FOIS’98, Trento, Italy, June 6-8,1998), ed. by N. Guarino. Amsterdam: IOS Press, 1998, pp. 3–15.
[5] A. V. Smirnov, M. P. Pashkin, N. G. Shilov and T. V. Levashova,”Ontologies in Artificial Intelligence Systems. Part 1” in ArtificialIntelligence News, #1, 2002, pp. 3–13 (in Russian).
[6] G. S. Plesniewicz, V. B. Tarassov, B. S. Karabekov and NguyenThi Min Vu, Methods and Languages for Ontological Modeling.Almaty: IICT MRS KR Editions, 2017 (in Russian).
[7] J. F. Sowa, ”Top-Level Ontological Categories” in InternationalJournal of Human-Computer Studies, vol. 43, 1995, pp. 669–685.
[8] R. Hodgson et al., The NASA QUDT Handbook, 2014.[9] W. Kuhn, ”A Functional Ontology of Observation and Measure-
ment” in Proceedings of the 3rd Workshop on GeoSemantics(GeoS’2009, Mexico City, December 3–4, 2009). Lecture Notesin Computer Science, vol.5892. Berlin: Springer-Verlag, 2009,pp. 26–43.
[10] ISO/IEC Guide 98-3: 2008. Uncertainty of Measurement. Part 3.Guide to the expression of uncertainty in measurement (GUM:1995). Geneva: JCGM, 2008.
[11] GOSR 54500.1–2011/Guidebook ISO/IEC 98-1: 2009. Measure-ment Uncertainty. Part 1. Introduction to Measurement Uncer-tainty Guide. Part.3. Guide to the Expression of Uncertaintyin Measurement. Moscow: Standartinform Editions, 2012 (inRussian).
[12] V. B. Tarassov, A. P. Kalutskaya and M. N. Svyatkina, ”Granular,Fuzzy and Linguistic Ontologies Enabling Mutual Understandingof Cognitive Agents” in Proceedings of the 2nd International Sci-entific and Technical Conference on Open Semantic Technologiesfor Inteligent Systems (Minsk, BSUIR, February 16–18, 2012).Minsk: BSUIR Editions, 2012, pp. 267–278 (in Russian).
[13] T. Buzan and C. Griffiths, Mind Maps for Business.Using theUltimate Business Tool to Revolutionise How You Work, 2nd ed.New York: Pearson, 2014.
[14] A. S. Doynikov, ”Classification of Measured Properties” in Pro-ceedings of the 5th International Conference on Soft Comput-ing and Measurements (SCM’2002, St.Petersburg, St.PetersburgState Electrotechnical University LETI, June 25–27, 2002).St.Petersburg: LETI Editions, 2002, pp. 46–49 (in Russian).
[15] G. B. Burdo, E. V. Vorobyeva, ”Ontological Approach in Design-ing Technological Processes” in Open Semantic Technologies forIntelligent Systems, 2015, pp. 461–464 (in Russian).
[16] A. E. Fridman, Fundamentals of Metrology. Modern Course.St.Petersburg: SPU Professional Editions, 2008 (in Russian).
[17] L. A. Zadeh, ”Toward a Theory of Fuzzy Information Granulationand its Centrality in Human Reasoning and Fuzzy Logic” inFuzzy Sets and Systems, vol.90, 1997, pp. 111–127.
[18] L. Reznik, ”Measurement Theory and Uncertainty in Measure-ments: Application of Interval Analysis and Fuzzy Set Methods”in Handbook of Granular Computing, ed. by W. Pedrycz, A.Skowron and V. Kreinovich. Chichester: John Wiley and SonsLtd, 2008, pp. 517–532.
[19] V. B. Tarassov, ”Granular Measurement Structures in AmbientIntelligence: Vasiliev’s and Belnap’s Sensors and Their Com-munication Models” in Information-Measurement and ControlSystems, #5, 2013, pp. 65–74 (in Russian).
[20] A. N. Borisov, A. V. Alexeev, O. A. Krumberg et al. Decision-Making Models on the Basis of Linguistic Variables. Riga:Zinatne, 1982 (in Russian).
ОБ ОНТОЛОГИЧЕСКОММОДЕЛИРОВАНИИИЗМЕРЕНИЙ В КОМПЛЕКСНОЙ СИСТЕМЕМОНИТОРИНГА ТЕХНИЧЕСКОГО ОБЪЕКТА
Королева М. Н., Бурдо Г. Б.
Рассмотрена комплексная проблема мониторингасложного технического объекта (на примеремостовогоперехода), описаны особенности организации и задачиизмерений в системе мониторинга. В интересах пред-ставления и систематизации знаний об измерениях из-ложен онтологический подход к спецификации изме-рений, предложена иерархическая система онтологийизмерений, построены основные онтологии измеренийнижнего уровня с помощью ментальных карт. Особоевнимание уделено описанию видов неопределённостив измерениях.
Received 10.01.19
246
Attributes, scales and measures for knowledgerepresentation and processing models
Valerian IvashenkoBelarussian State University of Informatics and Radioelectronics
Minsk, [email protected]
Abstract—The system of measures and features forscaling and ranking knowledge processing phenomena isconsidered. Some types of measurement scales were gen-eralized. Such attributes and measures as key elements ofthe knowledge representation language and the distancebetween the texts of such languages were considered to-gether with others combining means of the set theory,ordered sets and the theory of formal languages. Theproposed concepts are towards the integration of knowledgeprocessing models, including artificial neural networks.
Keywords—semantic networks, knowledge representa-tion, knowledge processing, scales, features, measure, mea-surement
I. INTRODUCTION
The purpose of the article is to get answers to thefollowing questions:• What are the types of scales [1], [2] are and their
features (attributes)?• How complex is the measurement scale for an
arbitrary set of features?• What features can be mined from knowledge repre-
sentation [3], [4] models?• What features can be mined from information pro-
cessing [3]–[6] models and knowledge processingphenomena [3], [4], [7]?
Objects are designated by signs in the order of perceptionprocesses for the representation of knowledge. The be-coming of signs in these processes allows to investigatethe properties and attributes of objects.
II. MEASUREMENT SCALES AND FEATURES
From a mathematical point of view, a feature is definedby a function that is defined on a set of objects andallows for each of them to get a particular value ofthe feature. Each feature with relational structures [4]or models on a set of objects and a set of values ofthis feature form a scale. The relational structure ormodel allows to structure a set of objects or a set ofvalues of the feature. Depending on the complexity,scales vary by types. The complexity of the scale isdetermined by power of the model carrier set and itsstructure. One of the ways to set the scale structure isto order the set of values of the feature. In this case, thesignature of the corresponding model contains a binary
relation of a reflexive order [8], which has the propertiesof antisymmetry and transitivity. If the order is trivial(has the property of symmetry), then the scale is callednominal. Another important type of scale with an orderrelation is the (linear) ordinal scale, the order on whichhas the property of linearity. An important type of scalesand features (attributes) and are quantitative scales andfeatures which values are numbers. Often these are scaleswith a linear order. Quantitative attributes (measures)allow measurements. Within the scale, the values of onefeature can be considered as objects that may have theirown features. Thus, a sequence of scales can be built,reflecting one set of features and their models to the nextones. One of the quantitative scales in such sequencesand the corresponding features are the scale and thefeature that measures the number (power of the set)of the mapped objects for any value of the feature inanother scale. The model of feature values in this scaleis in one-to-one correspondence with a well -ordered setof cardinal numbers. Such a feature as modishness isassociated with this scale and order on it:
supγ∈Σ
(∣∣∣∣argχ∈Σ
(〈s (χ) , s (γ)〉)∣∣∣∣)
(1)
Here arg means a function that returns a subset ofobjects of a set Σ, the value of one attribute α of whichis equal to the value of another attribute β
argγ∈Σ
(〈α (γ) , β (γ)〉) =
= γ |(γ ∈ Σ) ∧ (α (γ) = β (γ)) (2)
Under the mods are understood the elements of theset of objects of the primary scale, the feature valuesof which are not less than modishness. If there are nosuch elements, the modishness is external, otherwise itis internal.
argζ∈Σ
(⟨∣∣∣∣argχ∈Σ
(〈s (χ) , s (ζ)〉)∣∣∣∣ , supγ∈Σ
(∣∣∣∣argχ∈Σ
(〈s (χ) , s (γ)〉)∣∣∣∣)⟩)
(3)The projection (mapping) of the original scale on a
non-empty set of modes forms a new scale (subscale),all objects in which are modes.
maxχ∈Σ
(α (χ)) = max (α (χ) |χ ∈ Σ) (4)
247
It is natural to distinguish scales and attributes by thenumber of values of a attribute in a scale, accordingto which attributes and scales can be finite, includingbinary attributes and scales, scales with n values, andinfinite ones. In addition to modes in scales with anorder relation, you can select the medians, the set ofall medians in the scale forms a medianoid.
Section ϕ on a scale ψ:
O (〈ϕ,ψ〉) =
= (∀χ (∀γ ((〈χ, γ〉 ∈ ϕ)→ (¬ (ψ (χ) ≥ ψ (γ))))))(5)
The lower sections of the set Σ on the scale ψ:
LS (〈Σ, ψ〉) =
=
〈α, β〉
∣∣∣∣(
(O (〈α× β, ψ〉) ∧ (β ⊂ Σ))∧((α = Σ/β) ∧ (|α| ≤ |β|))
)(6)
LS (〈Σ, ψ〉) =
=
〈α, β〉
∣∣∣∣(
(O (〈α× β, ψ〉) ∧ (β ⊂ Σ))∧((α = Σ/β) ∧ (|α| ≤ |β|))
)(7)
The upper sections of the set Σ on the scale ψ:
US (〈Σ, ψ〉) =
=
〈α, β〉
∣∣∣∣(
(O (〈α× β, ψ〉) ∧ (α ⊂ Σ))∧((β = Σ/α) ∧ (|α| ≥ |β|))
)(8)
Embedding sections ϕ:
C (ϕ) =
=
〈〈α, β〉 , 〈γ, δ〉〉
∣∣∣∣(
((α ⊆ γ) ∧ (δ ⊆ β))∧(〈〈α, β〉 , 〈γ, δ〉〉 ∈ ϕ)
)
(9)
Medianoid embedding sections ϕ:
M (ϕ) =
=
µ
∣∣∣∣(∀β(∀γ (∃α (∃δ (〈〈α, β〉 , 〈γ, δ〉〉 ∈ ϕ)))→ ((µ ∈ β) ∼ (µ ∈ γ))
))
(10)
Medianoid set Σ on the scale ϕ:
M(C((LS (〈Σ, ϕ〉) /US (〈Σ, ϕ〉))×(US (〈Σ, ϕ〉) /LS (〈Σ, ϕ〉)))) (11)
A scale [1] is called quantitative if the values of theattribute in it are numbers. If the numbers are real, thenthe scale will be called charged [9], [10]. A chargedscale, the numbers in which are non-negative, will becalled the measured scale [1], [2]. The feature values inthe scale can also be elements of a module or a vectorspace. If a pseudometric or metric is defined on a setof values of a feature in a scale, then the correspondingscales will be called pseudometric or metric [2], [13]. For
them, the concept of medoid can be defined dependingon the measures α and β.
argζ∈Σ
(⟨α (〈Σ, ζ〉) , inf
γ∈Σ(β (〈Σ, γ〉))
⟩)(12)
The measure β is usually the same as α, which hasthe following form:
α (〈Σ, γ〉) =∑
χ∈Σ
µ (〈χ, γ〉) (13)
a measure can taked as α (〈Σ, γ〉) = supχ∈Σ
(µ (〈χ, γ〉)) or
α (〈Σ, γ〉) = |L (〈Σ, γ〉)|, where L is the function of theremote set of a point χ in the set Σ:
L (〈Σ, χ〉) =
= γ |((σ ∈ S (〈Σ, χ, γ〉)) ∧ ((σ ≥ 0) ∧ (γ ∈ Σ)))(14)
and S is the function of distancing the point χ from thepoint γ on the set Σ:
S (〈Σ, χ, γ〉) =
=
σ
∣∣∣∣∣∣
σ =
∑
ζ∈Σ
(µ (〈χ, ζ〉)− µ (〈γ, ζ〉))
(15)
Also, a medoid can be selected based on distancing at
β (〈Σ, γ〉) = 1. (16)
When considering an arbitrary set of features definedon a set of objects, it is possible to move from this setto a single multiple feature according to the scheme:
∣∣∣∣ ×i∈Γ
(PiO)∣∣∣∣ =
∣∣∣∣∣
(×i∈Γ
Pi
)O∣∣∣∣∣ (17)
There is possible reverse transition. If the sourcefeatures are binary, then the multiple feature has a rangeof power 2. Any n-ary feature can be presented as amultiple feature with the range of the same power 2|Γ|.It should be noted that a countable number of binaryfeatures for a finite set of objects corresponds to anelement of uncountable set of multiple features. Thefinite multiple features of binary features and a set ofobjects generate the formal context [11] for which thelattice of formal concepts [12] can be constructed. Thecorresponding indicator sets in rows or columns of theformal context can be interpreted as elements of a vectorspace [13] of finite dimension over a finite field F2 [14].Thus, the lattice can be extended to a vector space.The same is true for any n ary features when n is aninteger power of a prime number. Thus, a finite formalcontext can be spliced onto a set of finite vector spaces(a pseudometric and pseudonorm can be introduced).
The graph of the lattice of formal concepts [11], [15]can be considered as the carrier of a metric space. Also,a finite formal context can be mapped into a set of finitemodules that can be mapped to a non-infinite module ofintegers [16].
248
III. KNOWLEDGE REPRESENTATION LANGUAGES, ITSTEXTS AND FEATURES
Languages [17], [18] will be called straight languagesiff each text of which does not contain its componentsmore than once. By analogy with symmetric (symmet-rical) languages [3], languages containing as strings alllines that are the result of cyclic permutations [19] ofany other text of this language will be called cycliclanguages. In a similar way, inverted (palindromic), di-hedral and alternating [19] (antisymmetric) language areconsidered.
In processes and phenomena, frequently occurringfragments (sub-phenomens and substrings) are distin-guished. Frequent repetitions of adjacent identified phe-nomena are recorded and fixed as meta-events. The ratioof the presence or absence of a fragment in a phe-nomenon is an attribute. Each designation is associatingwith a coordinate vector in an infinite-dimensional spacewith a metric introduced on the set of these vectors.
On the set of texts such features as key elements areinvestigated. Consider the texts of some sublanguage Lof the language U in the alphabet A. Let’s set text τtransformation into functional form:
ϕ (τ) = 〈i, τi〉 |(i ≤ dim (τ)) ∧ (i ∈ N) (18)
Least powerful sets:
N (〈γ, χ〉) = (∀α(∀β(((α, β ⊆ χ) ∧ (α ⊂ β))→(∃δ ((δ ∈ γ) ∧ (δ ⊆ α) ∧ (|α/δ| = |β/α|))))))L (〈γ, χ〉) = (χ ⊆ γ) ∧N (〈γ, χ〉)∧(∀δ ((δ ∈ γ)→ (∃α ((α ∈ χ) ∧ (α ⊆ δ)))))
(19)
Set of combinations of ε-covering σ-elements:
D (〈α, γ, ε, σ〉) = (∀χ(∀λ(((χ ∈ λ) ∧ (λ ∈ γ)) ∼(∃β(
(β ⊆ α) ∧ (|β| ≥ ε) ∧(χ ⊆ σ ∩⋂δ∈β δ
))))))
(20)Key rank (ϕ-rank):
X (〈α, β, γ, σ, ϕ〉) =max (ε |(D (〈α, γ, ε, σ〉) ∧ ϕ (〈α, β, γ〉))) (21)
Extra key combinations (ϕ-combinations):
F (〈α, β, σ, ϕ〉) = χ|∃γ(L (〈γ, χ〉)∧D (〈α, γ,X (〈α, β, γ, σ, ϕ〉) , σ〉) ∧ ϕ (〈α, β, γ〉)) (22)
Splitting combinations:
E (〈α, γ〉) = (∀χ ((χ ∈ α)→ (∃λ ((λ ∈ γ) ∧ (λ ⊆ χ)))))I (〈β, γ〉) = (∀χ (∀λ (((χ ∈ β) ∧ (λ ∈ γ))→ (¬ (λ ⊆ χ)))))T (〈α, β, γ〉) = E (〈α, γ〉) ∧ I (〈β, γ〉)
(23)Key combinations (ϕ-combinations):
R (〈α, β, σ, ϕ〉) =γ |(γ ∈ F (〈α, β, σ, ϕ〉)) ∧ (¬G (〈α, β, σ, ϕ, γ〉)) (24)
where
G (〈α, β, σ, ϕ, γ〉) = (∃χ(∃λ (χ ∈ λ)∧(λ ∈ γ) ∧ (λ/ χ ∪ (γ/ λ) ∈ F (〈α, β, σ, ϕ〉))))
(25)Key schemes (ϕ-schemes):
S (〈α, β, σ, ϕ〉) =⋃
γ∈R(〈α,β,σ,ϕ〉)γ (26)
External key schemes (ϕ-schemes):
SE (〈α, β, σ, ϕ〉) = δ| (δ ∈ S (〈α, β, σ, ϕ〉))∧(∀χ ((χ ∈ β)→ (χ ∩ δ = ∅))) (27)
Internal key schemes (ϕ-schemes):
SI (〈α, β, σ, ϕ〉) = χ ∩ δ| (δ ∈ S (〈α, β, σ, ϕ〉))∧((χ ∈ β) ∧ (∅ ⊂ χ ∩ δ)) (28)
Let define
W (Γ) = λ|(∃χ((χ ∈ Γ) ∧ (∀ι((ι ∈ N)→(λ (ι) = χ
(ι+ min
(⋃γ∈χ γ1
)−min (N)
))))))
(29)and
U (Γ) =⋃γ∈Γ γ
V (Γ) =⋃γ∈Γ γ2 (30)
Key phrases (ϕ-phrases):
P (τ) = W (S (τ)) (31)
where τ = 〈U/L,L,A, T 〉 with universal language U , selected(sub)language L with alphabet A and predicate T .
External (PE) and internal (PI) key phrases are defined asfollows:
PE (τ) = W (SE (τ))PI (τ) = W (SI (τ))
(32)
Key components (C) and its external (CE) and internal (CI)ones are defined in similar way:
C (τ) = U (S (τ))CE (τ) = U (SE (τ))CI (τ) = U (SI (τ))
(33)
Finally, key elements (E) as well as external (EE) andinternal (EI) components are:
E (τ) = V (C (τ))EE (τ) = V (CE (τ))EI (τ) = V (CI (τ))
(34)
Key schemes, phrases, components, elements and theirsets, are features of languages [18] and families ofphenomena [3]. Other features of texts are: length, period[20], etc. Other features can be obtained by correlatingtexts of languages and texts in languages obtained bycanonization, symmetrization [3], [19], circulation, loop-ing, etc. The investigation of texts and associations withkey elements allows to explore the model semantics ofthese key elements [3].
For comparing texts of languages [17], [18] and phe-nomena [3], it is possible to use scales of distant vectors,whose components correspond to the number of deleted,added, rearranged, duplicated or merged components ofa particular class in the texts. In turn, the norm of distantvectors corresponding to the metric on the metric scalecan be calculated using the metric operator. The textsof languages and phenomena are mapped respectivelyon this scale. These and other attributes correspond tothe functions defined on the relations of the knowledgespecification model [5].
249
IV. FORMAL MODELS AND PHENOMENA OFKNOWLEDGE PROCESSING
Among the formal information processing models [5],[6], it is possible to distinguish elementary formal infor-mation processing models that have only one operation.Any formal information processing model can be reducedto an elementary one, by combining all its operationsinto one operation of an elementary model. It is possibleto investigate the operational semantics of texts of thelanguage representing the states of the formal informa-tion processing model in addition of the investigationof model semantics of key elements of this language. Inparticular, the model set-theoretic semantics is associatedwith the (implicit) operations of choosing (marking)the notation of a set and searching (marking) of theelements of this set. The study of operational semanticsis related to discovering the rules and properties ofbecoming elements of texts and phenomena. The basisfor the study of operational semantics is both the keyelements of the language themselves and the multiplefeatures, the distant vectors of their set and averagingbetween the texts before and after the application of theoperation. Other features can be obtained by examiningthe complements or inversions of the model (modelswith operations complementing the original operationsbefore the full operation (relation) or inverse them). Forcomparing formal information processing models [5],[6], [21], projections or the reduction of initial modelsto elementary formal models of information processingand features defined between them can be used. Firstof all, it is necessary to find operations that are thesmallest symmetric differences of operations isomorphicto the operations of the compared models taking intoaccount the maximum possible one-to-one fusion of thestates of these models. Then, it is necessary to choosethe one among these operations having the minimalaveraging of the set of distant vectors between the textsbefore the application of this operation and after. Thisaveraging or monotonously (or linearly) dependent on itvalue can be an analogue of the distant vector betweenthe two models. If the models are not elementary, thenthey should be supplemented (if it is necessary) withempty operations and divided into one-to-one elementarymodels with preservation of one-to-one identification ofthe states so that the averaging of distant vectors betweenthem is minimal. For finite models (finite automata) thisproblem is solvable.
V. CONCLUSION
The results of this work allow to generalize the conceptof the scale and some of the features defined on them. Inaccordance with the knowledge specification model [5], anapproach is also considered to identify key elements of textsof languages, phenomena and operations, their semantics andmetric properties as the basis for scaling and analyzing theattributes of texts of languages and phenomena of informationprocessing models in intelligent systems.
REFERENCES
[1] Stevens, S. S. On the Theory of Scales of Measurement. Science, 1946,103 (2684), pp. 677 —680.
[2] Wolman, Abel G. Measurement and meaningfulness in conservation sci-ence. Conservation Biology, 2006, no 20, pp. 1626 –1634.
[3] Ivashenko, V. P. Ontologicheskaya model’ prostranstvenno-vremennykhotnoshenii sobytii i yavlenii v protsessakh obrabotki znanii [Ontologicalmodel of space-time relations for events and phenomena in the processingof knowledge]. Vestnik BrGTU, 2017, no 5(107), pp. 13 –17. (in Russian).
[4] Golenkov, V. V., et al. Ontology-based Design of Batch ManufacturingEnterprises. Otkrytye semanticheskie tekhnologii proektirovaniya intellek-tual’nykh system [Open semantic technologies for intelligent systems],2017, no 1, pp. 265––280.
[5] Ivashenko, V. P., Tatur, M. M. Printsipy platformennoi nezavisimosti i plat-formennoi realizatsii OSTIS [Cross-platform principles and principles ofplatform implementation of OSTIS]. Otkrytye semanticheskie tekhnologiiproektirovaniya intellektual’nykh system [Open semantic technologies forintelligent systems], 2016, pp. 145 –150. (in Russian).
[6] Kuz’mitskii, V. M. Printsipy postroeniya grafodinamicheskogo paral-lel’nogo assotsiativnogo komp’yutera, orientirovannogo na pererabotkuslozhnostrukturirovannykh znanii [Principles of constructing a graph-dynamic parallel associative computer oriented on processing complexlystructured knowledge]. Intellektual’nye sistemy [Intelligent systems], 1998,no. 1, pp. 156 –166. (in Russian).
[7] Golovko, V. A., Golenkov, V. V., Ivashenko V. P. et al. Integration ofartificial neural networks and knowledge bases. Otkrytye semanticheskietekhnologii proektirovaniya intellektual’nykh system [Open semantic tech-nologies for intelligent systems], 2018, no 2, pp. 133 –146. (in Russian).
[8] Davey, B. A., Priestley, H. A. Introduction to Lattices and Order (2nd ed.).Cambridge University Press, 2002, 312 p.
[9] Alexandroff, A. D. Additive set-functions in abstract spaces II. Mat.sbornik, 1941, vol. 9(51), no 3, pp. 563 –628.
[10] Landkof, N. S. Osnovy sovremennoi teorii potentsialov [Fundamentals ofthe contemporary theory of potentials]. Moscow, 1966, 516 p. (in Russian).
[11] Poelmans, J., Ignatov, D., Kuznetsov, S. et al. Fuzzy and rough formalconcept analysis: a survey. INT J GEN SYST, 2014, Vol. 43, no 2, pp. 105 –134.
[12] Ganter, B., Wille, R. Formal concept analysis – mathematical foundations.Berlin ; New York : Springer, 1999, 284 p.
[13] Shafarevich, I. R., Remizov, A. O. Lineynaya algebra i geometriya [Linearalgebra and geometry]. Moskva: Fizmatlit, 2009, 511 p. (in Russian).
[14] Lidl, P., Niderraiter, G. Konechnye polya. [Finite fields]. Moskva: Mir,1998, In 2 V. 430 p. (in Russian).
[15] Gratzer, G. J. Obshchaya teoriya reshetok [General lattice theory]. Moskva:Mir, 1982, 452 p. (in Russian).
[16] Van der Varden, B. L. Algebra. Moskva: Nauka, 1975, 623 p. (in Russian).[17] Markus, S. Teoretiko-mnozhestvennye modeli yazykov [Set-theoretic mod-
els of languages]. Moskva : Nauka, 1970. 332 p. (in Russian).[18] Handbook of Formal Languages. Vol. 1, Word, Language, Grammar.
Springer–Verlag, 1997, 873 p.[19] Vinberg, E. B. Kurs algebry [Algebra course]. Moscow: ”Factorial-Press”,
2001, 544 p. (in Russian).[20] Smith, B. Metody i algoritmy vychislenii na strokakh [Computing patterns
in strings]. Moskva: “I.D. Viliyams” [Moscow: P.H. Williams], 2006,496 p. (in Russian).
[21] Cortes, C., Gonzalvo, X., Kuznetsov, V., Mohri, M. and Yang, S. Adanet:Adaptive structural learning of artificial neural networks. arXiv preprintarXiv:1607.01097, 2016. Available at: https://arxiv.org/pdf/1607.01097.pdf(accessed 2017, Dec)
ПРИЗНАКИ,ШКАЛЫИМЕРЫДЛЯМОДЕЛЕЙПРЕДСТАВЛЕНИЯ И ОБРАБОТКИ ЗНАНИЙ
Ивашенко В.П.Рассмотрена система признаков и мер для шкалирова-
ния и ранжирования явлений обработки знаний. Средства-ми методов теории множеств, упорядоченных множеств итеории формальных языков рассмотрено обобщение шкалнекоторых видов, а также впервые приведено формальноеописание таких признаков и мер, как ключевые элементыязыков представления знаний и метрики на текстах этихязыков и моделях обработки информации. Предложенныепонятия ориентированы на интеграцию моделей обработкизнаний, включая искусственные нейронные сети.
Received 14.12.18
250
Control of a Technological Cycle of ProductionProcess Based on a Neuro-Controller Model
Viktor Smorodin, Vladislav ProkhorenkoFrancisk Skorina Gomel State University
Gomel, [email protected], [email protected]
Abstract—In this paper a method for constructing amodel of a neuro-controller for implementation of controlin the presence of external disturbances for the optimaltrajectory finding on the phase plane of system states fortechnological cycle of a production process is proposed.A type of a neuro-controller based on recurrent neuralnetwork architecture with long short-term memory blocksas a knowledge base on the external environment, previousstates of the controller and control actions is being used.
Keywords—neuro-controller, recurrent neural network,LSTM, adaptive control, technological cycle of productionprocess
I. INTRODUCTION
Recently, artificial intelligence theory is being widelyapplied to solving such classes of problems as classifica-tion, clustering, prediction, approximation, data compres-sion and other tasks [1][2][3][4]. However latest researchin this area shows, that the application of artificial neuralnetworks (ANNs), which are currently being consideredone of the most important research directions in the area,is not limited only to the listed classes of problems.Researchers and practitioners are being interested insolving problems of complex process control in the areasof activity which are difficult to formalize [5][6][7].
It should be noted that despite the high level ofcomplexity of the practical problems in this area thatcan be solved by application of the artificial intelligencemethods, ANNs are a fairly effective and convenienttool for finding solutions to these problems based onconstruction of a finite set of mathematical models,which is being considered as a single model of the objectunder study as a complex technical system [8].
For this reason application of neural networks train-ing for complex technological objects analysis providesan important advantage over the traditional researchmethods, including the simulation modeling [9], becauseduring the training process neural network is able toextract complex dependencies between input and outputdata, as well as provide necessary generalizations.
When analyzing the operation of complex technicalsystems the existing methods of analysis often provideinsufficient effectiveness, especially in cases of projectmodeling when the structure of such objects can bealtered in the process of their evolution. The reason
of this is in diversity and complexity of the practicaltasks arising at the stage of project modeling, and alsowhen estimating the operation reliability and safety forpotentially hazardous technical systems.
Therefore development of a new approach to analyzecomplex systems at the stage of their project modelingautomation, which would allow to take into accountthe changes in structural connectivity of the controllingsystem when changes of technological cycle structureoccur due to failures, is a task of great importance.
Such an approach can be developed using the pro-cedure of project modeling of the object under study,which is based upon an adaptable structure of the controlsystem using the neuro-controller model, which takesinto account all the changes in the technological cycleof production operation process.
It is known that the main task of the effective controlof the technological cycle of production consists inimplementing the sequence of universal control actionsthat would allow to optimize the output parameters ofthe technological system when possible changes in thestructure of the technological cycle occur. Such changescan be the result of having the elements of potentiallyhazardous production in the multicriterial control prob-lem under consideration.
The recent research in the area shows that high-qualityanalysis of the control systems operation requires takinginto account a great amount of factors, which undergochanges during the process of operation of the objectunder study. It can be achieved through implementingadaptive control algorithms for the systems under study.
The recent trends in the use of some system-wideprinciples and methods of research in various fields ofknowledge, open semantic technologies for intelligentsystems, lead to the unification of the system approachwhen considering specific scientific and practical prob-lems.
Such trends allow to hope for the future creation ofthe necessary knowledge base and the software capableof logical inference as part of the task under considera-tion, which would allow the researcher to interact withsystems of varying degrees of complexity, disregardingtheir physical nature or the limitations of some specific
251
formalization.One of the most important tasks in this area is the
task of constructing an adaptive control system fortechnological cycle, which is capable of providing arational structure of the control loop at the given momentof time [5]. The latter is directly related to the loop’srestructuring during its operation and constructing theadaptive control algorithms to optimize the technologicalcycle resource consumption in real time in the presenceof external control actions.
The neural network controller modeling is effectivewhen a high-quality controller of the controlled systemis available [11]. In this case the neural network acts asan approximator of its function and is trained to simulatethe effects of the controller on the controlled system. Insome situations it may be the case that the use of theneuro-controller constructed in that way is more practicalbecause of the common properties of ANNs.
Results in the area of research of controlled tech-nological systems based on constructing the models ofANNs for providing effective control of the technologicalproduction cycle [10][12] are given in this paper. Amethod for constructing a model of neuro-controller fortechnological production cycle control in the presence ofexternal disturbances is being proposed.
Implementation of control for the optimal trajectoryfinding task in an arbitrary region of complex structurerequires a high-quality controller, which is able to adaptits actions according to the local environmental dataavailable at the given moment of time. To implementsuccessfully pathfinding strategies it is also required tostore and take into account the data received by thecontroller at the previous moments of time. Explorationof applicability of ANNs for solving the tasks of thisclass is an important research direction because of theadvantages that these models have.
In this paper a method for constructing a model ofa neuro-controller for implementation of control for theoptimal trajectory finding in the case of a dynami-cally changing region of arbitrary configuration is beingproposed. Recurrent neural network architecture withLSTM-blocks, which allows to store information aboutthe states of the system at the past moments of time thatmay be significantly distant from the current moment oftime [13][14], is being used as a mathematical model.
II. RELATED WORK
ANNs have proven to be an effective instrument tosolve a set of various problems from different areasof human action. The properties of the ANNs maderesearchers to consider ANNs as a suitable model tosolve control tasks. Different approaches were devel-oped to implement neural networks in the control tasks[24][26][27] and many examples of successful appli-cations exist [5][6][7][28]. Applications in the area of
production process control and optimization were alsodeveloped, typically using feedforward types of neuralnetworks in order to solve specific tasks related to theproduction operation or its aspects [25][29][31]. Someadaptive control approaches based upon neural networkmodeling were proposed for plant control and dynamicsystems control [30][26].
Recurrent neural networks research shows that it maybe useful to apply such architecture to the tasks whereprocesses evolving over time take place. The recur-rent neural network architecture is capable of capturingtime dependencies therefore allowing to solve variousreal-world tasks [3][4]. However while having interest-ing potential capabilities [3][15] that can be achievedwith different variations of the recurrent architecture[7][16][17][18][19], it also has a known problem whenthe task requires taking into account the long-term depen-dencies [14]. LSTM blocks allow the long-term storageof data [13] and can be applied to the tasks wherelong-term time dependencies must be taken into account[20][21][22][23].
III. FORMALIZATION OF THE TASK
In the considered task of trajectory finding on thephase plane of system states, the controlled object movesacross a two-dimensional region which is divided intononintersecting subregions (cells) that may be passableor impassable. Cells beyond the edge of the regionare considered to be impassable. A passable subregionis assigned a value of 0, while the impassable one isassigned a value of 1. In the given region, a targetsubregion is designated. It is guaranteed that a path fromthe starting position of the controlled object to the targetsubregion exists in the region at any stage of its evolution.At each moment of time, the controller receives a vectorof seven elements: data on four cells adjacent to thecurrent position of the controlled object, the distanceto the target subregion and the direction to the targetsubregion.
The result produced by the neuro-controller at a givenmoment of time is a four-element vector that determinesthe direction of the next move of the controlled objectin the region. The controlled object continues to moveuntil the target subregion is reached.
IV. ARCHITECTURE OF THE NEURO-CONTROLLER
The set of specific features of the control tasks, whichrequire controller to make decision within some long-term strategy in the case of a dynamically changing envi-ronment and availability of the local environmental dataof arbitrary nature at the given moment of time, requiresthe controller to have a specific structure. A structure ofthe controller, that includes encoder module for encodingand pre-processing of the environmental data, memorymodule for the long-term storage of data and decision-making module, which determines the output signals of
252
the the controller at the given moment of time, is beingproposed. The proposed general scheme of the controlleris shown in Figure 1.
Figure 1. General scheme of the controller
This implies from the point of view of the ANNs ar-chitecture that the network will have structural elementswith the functions that can be interpreted as functionsof the listed modules. The input layers of a given neuralnetwork can be considered as the encoder module whichpreprocesses and encodes input signals. (For example,in the case when the controller has image as input data,a sequence of convolutional and subsampling layers thatgradually reduces dimensionality of the data and convertsit into a vector, can be considered as encoder module.) Asubnet that consists of LSTM modules can be consideredas the structural element for long-term data storage. Asubnet of arbitrary structure that is connected to thestructural elements for encoding and storing data andincludes the output layer of the network, that producesthe control signals, can be considered as the decision-making module.
In the framework of the described approach a neuro-controller with a recurrent architecture that containsLSTM blocks is being considered in this paper. Therecurrent architecture with LSTM block includes threefully-connected layers consisting of five, sixteen and fourneurons, respectively. The LSTM block has a state ofsize 16 and is connected to the second layer of theneural network through the elements of a time delay.Its current state is passed to the input of the third layer.There is also a feedback connection through a time delayelements between the second layer and the first layer.The architecture was selected experimentally as the onethat would have the minimal number of neurons in alllayers and be able to train and perform pathfinding onthe testing set. In Figure 2 the scheme of this architectureis shown.
The choice of the recurrent architecture is based uponthe necessity to take into account time dependencies inthe environmental data available to the controller. LSTMblocks allow the long-term storage of data. In case of thepathfinding task it is necessary for the implementation of
the pathfinding strategies stretched upon relatively longperiods of time, required by the task.
The neuro-controller model, the training and testingenvironments, and data generation process were imple-mented in Python programming language using Tensor-flow machine learning framework.
Figure 2. Scheme of the recurrent neural network architecture withLSTM block.
V. GENERATING TRAINING DATA
In order to train a neural network successfully it isimportant to use a large sample of data that adequatelyrepresents a variety of real-world situations that can beencountered by the neural network.
The neuro-controller described in this paper is used tosolve the task of pathfinding in a complex environmentof arbitrary structure that can change dynamically overtime. Therefore examples of such environments need tobe generated for training and testing.
30x30 regions with random placements of impassablesubregions were generated for the training procedure.Cellular automaton has proven to be a suitable modelwhich allows implementing a gradual evolution of thestructure of the region. Parameterizing the automata indifferent ways it is possible to achieve various patternsof structural change, which will result in increase ordecrease of the amount of impassable cells in the regionover time, or have circular nature. The evolving regionscan be randomized further by selecting the lengths oftime periods (steps) in which the next change to theregion will happen.
100,000 sequences of regions of 30x30 cells withimpassable areas changing over time were generated tobe used in the training and testing process of the neuro-controller. In Figure 3 example of a region evolvingthrough time is shown.
In this paper supervised learning was used to train theneuro-controller. A recurrent neural network is trained onsequences of input and output signals. In order to trainthe neuro-controller to implement pathfinding strategiesthe sequences have to be of significant length. Sequences
253
of 40 movements were used for the training procedure.A sequence consists of a list of vectors of the localdata for the current cell in the path (network inputs,including data on the adjacent cells, calculated distanceand direction to the target cell) and a list of correspondingvectors of required movement to achieve the next cell inthe path (desired network outputs).
Such sequences in the task considered can be obtainedby generating example paths in the regions. For thispurpose in each region a starting cell and a target cellwere selected randomly. In order to be able to obtainthe training sequences of the required length of 40movements it was checked that throughout the region’sevolution the path between the cells existed and that theshortest path between them was at least 40 movementslong. The described procedure of cells selection wasrepeated several times in each region. Sometimes theconfiguration of the region and evolution in its structuremade it impossible to select suitable cells. 10% of theregions with suitable cells were used for testing after thetraining was complete.
The best-first search was used to generate the pathsbetween the selected cells pairs in the evolving regions.Considering the non-static nature of the regions and thefact that only local data is available at each moment oftime to the neuro-controller, the paths were generateddynamically. A path was regenerated started with eachpoint where a change in local data was triggered by theregion’s structure dynamic changes. Based on the lengthof the paths generated by the described procedure one ormore training sequences were prepared based on each ofthem.
60,000 training sequences of 40 movements wereobtained based on the generated paths and used fortraining.
VI. TRAINING THE NEURO-CONTROLLER
During the training process such values of the networkparameters (connection weights and bias values of neu-rons) are found that the network produces desired outputsfor the given inputs. Training can be considered a non-linear optimization task of minimizing some loss functionspecified on the training set with respect to all of thenetwork parameters. In this paper the supervised learningwas used, which corresponds to the situation when alarge dataset with examples of the control sequences isavailable.
The neuro-controller was trained using the RMSPropoptimization algorithm to minimize the loss function.The cross entropy function was used as the loss function.The training set of example sequences was divided intobatches and the parameters of the neural network (allweights and neuron biases) were corrected after pre-senting a batch of 50 sequences. Figure 4 shows theminimization of the loss function during first 20 epochsof training.
Figure 3. Example of a dynamically changing over time regiongenerated for the neural network training.
254
Figure 4. The loss function values during first 20 epochs of trainingof the neuro-controller.
VII. TESTING THE TRAINED NEURO-CONTROLLER
After the training the neuro-controller was able tofind trajectory successfully on the phase plane of thecontrolled system states with dynamically changing con-figuration of the allowed states subregion. 10% of thegenerated region sequences were used to perform thetesting and assess the performance of the trained neuralnetwork. A test was considered to be successful if theneuro-controller did not perform any forbidden actions(moving on impassable cell) and was able to reach thetarget cell in less than 60 movements. The controllerwas able to generate path to the target subregion in areasonable time in approximately 70% of the regions inthe test set. Figure 5 shows an example of pathfindingby the neuro-controller.
VIII. CONCLUSION
The theoretical research results described in this pa-per provide a basis for the future development of neweffective methods of analysis and synthesis of optimalstructure of the technical systems with adaptive control.The approach proposed by authors is applicable withinits framework to a whole variety of problems of theoptimal control structure synthesis and complex techno-logical systems synthesis. The research results can beused in the development of intelligent decision supportsystems designed for the corresponding tasks, automationof the technological production processes by artificialintelligence systems, development and automation of thedesigning process of new technological objects, and alsoquality assessment of the production technological cyclecontrol in real time.
In the course of this work an approach to applicationof the neuro-controller to implementation of the adaptivecontrol of the technological cycle was developed andtested. The experimentation on models has shown that
Figure 5. Example of successful pathfinding by the trained neuro-controller in a dynamically changing region.
255
neuro-controller based on a recurrent neural network withLSTM blocks can be successfully used for the adaptivecontrol tasks. LSTM blocks allow the neural network tostore information about the states of the system fromthe past moments of time that may be significantlydistant from the current moment of time, which allowsthe neural network to learn long-term dependencies andto reproduce long sequences of reactions to randomdisturbances and external influences. The possibility ofincreasing the efficiency of the existing architectureby adding additional memory modules and training onlonger data sequences depends on the specific parametersof the modeling object operation.
REFERENCES
[1] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard,W. Hubbard, L. D. Jackel Backpropagation applied to handwrittenzip code recognition. Neural Computation, 1998, vol. 1, no 4, pp.541-551/
[2] G. Cybenko Approximation by superpositions of a sigmoidalfunction. Mathematics of Control, Signals and Systems, 1989, vol.2, no 4, pp. 303–314/
[3] Multiple Object Recognition with Visual Attention. Available athttps://arxiv.org/abs/1412.7755 (accessed 2018, Nov)
[4] A. Karpathy Deep Visual-Semantic Alignments for GeneratingImage Descriptions. IEEE Transactions on Pattern Analysis andMachine Intelligence, 2017, vol. 39, no 4, pp. 664–676/
[5] K. Nouri, R. Dhaouadi Adaptive control of a nonlinear DC motordrive using recurrent neural networks. Applied Soft Computing,2008, vol. 8, no 1, pp. 371–382/.
[6] H. Mayer, F. Gomez, D. Wierstra, I. Nagy, A. Knoll, J. Schmid-huber A System for Robotic Heart Surgery that Learns to TieKnots Using Recurrent Neural Networks. IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 2006, vol. 22, no13, pp. 543–548/
[7] D. Huh, E. Todorov, Real-time motor control using recurrent neuralnetworks. Proceedings of IEEE Symposium on Adaptive DynamicProgramming and Reinforcement Learning (IEEE ADPRL), 2009,pp. 42–49/
[8] I.V. Maximey, O.M. Demidenko, V.S. Smorodin, Problems oftheory and practice of modeling complex systems, Gomel, F.Skorina State University, 2015, 263 p.
[9] I.V. Maximey, V.S. Smorodin, O.M. Demidenko, Development ofsimulation models of complex technical systems, Gomel F. SkorinaState University, 2014, 298 p.
[10] V.S. Smorodin, V.A. Prokhorenko The use of intelligent tech-nologies in optimizing the structure of controlled technologicalsystems. Reports of BSUIR, 2018, vol.8, no 118, pp. 30-35/
[11] O. Omidvar, D.L. Elliott Neural Systems for Control, New York,Academic Press, 1997.
[12] V. Smorodin, V.Prokhorenko Application of Neuro-ControllerModels for Adaptive Control. Chertov O., Mylovanov T., Kon-dratenko Y., Kacprzyk J., Kreinovich V., Stefanuk V. (eds) Re-cent Developments in Data Science and Intelligent Analysis ofInformation. ICDSIAI 2018. Advances in Intelligent Systems andComputing, 2019, vol 836, pp. 30-38/
[13] S. Hochreiter, J. Schmidhuber Long Short-Term Memory. NeuralComputation, 1997, vol 9, no 8, pp. 1735-1780/
[14] Y. Bengio, P. Simard, P. Frasconi Learning long-term dependen-cies with gradient descent is difficult. IEEE Transactions on NeuralNetworks, 1994, vol. 5, no 2, pp. 157-166/
[15] H.T. Siegelmann Computation Beyond the Turing Limit. Science,1995, vol. 268, no 5210, pp. 545–548/
[16] J. L. Elman Finding Structure in Time. Cognitive Science, 1990,vol. 14, no 2, pp. 179–211/
[17] M. I. Jordan, Serial Order: A Parallel Distributed ProcessingApproach. Advances in Psychology. Neural-Network Models ofCognition, 1997, vol 121, pp. 471–495/
[18] C. Goller, A. Kuchler Learning task-dependent distributedstructure-representations by backpropagation through structure.IEEE International Conference on Neural Networks, 1996, pp.347–352/
[19] Neural Turing Machines. Available athttps://arxiv.org/abs/1410.5401 (accessed 2018, Nov)
[20] P. Malhotra, L. Vig, G. Shroff, P. Agarwal, Long Short TermMemory Networks for Anomaly Detection in Time Series. ESANN2015 proceedings, European Symposium on Artificial Neural Net-works, Computational Intelligence and Machine Learning, 2015,pp. 89–95/
[21] A. Graves, A. Mohamed, G. Hinton Speech recognition with deeprecurrent neural networks. 2013 IEEE International Conferenceon Acoustics Speech and Signal Processing (ICASSP), 2013, pp.6645–6649/
[22] F. A. Gers, J. Schmidhuber, LSTM Recurrent Networks LearnSimple Context Free and Context Sensitive Languages. IEEETransactions on Neural Networks, 2001, vol. 12, no 6, pp.1333–1340/
[23] N. Tax, I. Verenich, M. La Rosa, M. Dumas Predictive BusinessProcess Monitoring with LSTM neural networks. Proceedings ofthe International Conference on Advanced Information SystemsEngineering (CAiSE), 2017, pp. 477–492/
[24] M.T. Hagan, H.B. Demuth Neural networks for control. Pro-ceedings of the American Control Conference, 1999, vol. 3, pp.1642—1656/
[25] A. Burduk Artificial Neural Networks as Tools for ControllingProduction Systems and Ensuring Their Stability. 12th Interna-tional Conference on Information Systems and Industrial Manage-ment (CISIM), 2013, pp. 487-498/
[26] K.S. Narendra, S. Mukhopadhyay Adaptive Control Using NeuralNetworks and Approximate Models. IEEE Transactions on NeuralNetworks, 1997, vol. 8, pp. 475-485/
[27] K.S. Narendra, K. Parthasarathy Identification and Control ofDynamical Systems Using Neural Networks. IEEE Transactionson Neural Networks, 1990, vol. 1, pp. 4-27/
[28] A. Broad, I. Abraham, T. Murphey, B. Argall tructured Neu-ral Network Dynamics for Model-based Control. Available athttps://arxiv.org/abs/1808.01184 (accessed 2018, Dec)
[29] Modeling a Neural Network Based Control for AutonomousProduction Systems. Available at http://www.sfb637.uni-bremen.de/pubdb/repository/SFB637-B2-10-002-IC.pdf (accessed2018, Dec)
[30] A. Venelinov Topalov, O. Kaynak Online learning in adaptiveneurocontrol schemes with a sliding mode algorithm. IEEE Trans-actions on Systems, Man, and Cybernetics, Part B, 2001, vol. 31,no 3, pp. 445—450/
[31] F.M. Dias, A.M. Mota Comparison between Different ControlStrategies using Neural Networks. 9th Mediterranean Conferenceon Control and Automation, 2001/
УПРАВЛЕНИЕ ТЕХНОЛОГИЧЕСКИМЦИКЛОМ ПРОИЗВОДСТВА НА ОСНОВЕ
МОДЕЛИ НЕЙРОКОНТРОЛЛЕРА
Смородин В.С., Прохоренко В.А.
Предложен способ построения модели нейрокон-троллера для реализации управления технологическимциклом производства при решении задачи поискаоптимальной траектории на фазовой плоскости состо-яний технологической системы в условиях наличиявнешних возмущений.Использован тип нейроконтрол-лера на базе рекуррентной нейросетевой архитектурыс модулями долгой краткосрочной памяти в качествебазы знаний о внешней среде, предыдущих состоянияхконтроллера и управляющих воздействиях на систему.
Received 28.12.18
256
Power Consumption for Autonomous WirelessSensor Network Nodes
Chaiko YelenaInstitute of Electrical Engineering and Electronics
EEF Riga Technical UniversityRiga, Latvia
Yelizaveta Vitulyova, Solochshenko AlexandrInstitute of space engineering and telecommunications
Almaty University of Power Engineeringand Telecommunications
Almaty, [email protected], [email protected]
Abstract—Environmental parameter using a large num-ber of spatially distributed wireless sensor network (WSN)nodes is an extensive illustration of advanced moderntechnologies, but high power requirement for WSN nodeslimits the widespread deployment of these technologies.Currently, WSN nodes are extensively powered up usingbatteries, but the battery has limitation of lifetime, powerdensity, and environmental concerns. Recently, WirelessSensor Networks (WSNs) have been widespread utilizedin automation. The power supply of WSNs has significantinfluences on their performance. This paper presents anovel power management circuit for WSNs. The proposednode needs to be able to send notifications to the utility,demanding the use of backup energy strategies. The authorsof the research offered an approach that can help touse solar-based energy harvester EH as the most viablesource of energy to be harvested for autonomous WSNnodes. Based on the test results, an integrated powercontrol module parameters are developed. The novelty ofthe research is an approach that includes different powersupply solutions in order to ensure defined signal transitionquality parameters for maximum effectiveness of WSNnodes power feeding.
Keywords—wireless sensor networks, WSN, solar energy,energy harvesting
I. INTRODUCTION
One of the most efficient and most widespread condi-tion monitoring methods is Wireless Sensor Network (WSN)monitoring. Wireless Sensor Networks (WSNs) play a keyrole, given to the fact that they constitute inherent distributedsystems, in which different platforms allow the inclusion ofmultiple analogue/digital input/output ports. A WSN is a dis-tributed network containing a large number of sensors and acontrol center unit which is able to monitor and control variousbehaviors of a structure or machinery. Multiple challengesmust also be handled, such as compact form factors, reducedenergy consumption, interference handling and variable nodedensity allocation. Adequate network operation and designrequire wireless channel analysis and optimization in order tominimize interference, energy consumption and enhance overallquality of service. WSN monitoring has many advantages overnon-networked systems, such as real-time and autonomousdata acquisition, enhanced data accessibility and intelligentdata analysis through intelligent algorithms [1]. This is ofparticular interest in the case of wireless sensor networks,given inherent restrictions in their operating conditions, as wellas in the potentially large number of nodes present in thenetwork. Utilizing WSN for factory line production monitoring
in [2] has leaded to lower operating costs and errors. UtilizingWSNs for performance monitoring in new energy generatorsas wind turbines and distributed solar panels, brings severaladvantages such as longer lifetime and lowering failures byreducing human involvement [3] and [4],[5]. Renewable energysources, such as solar radiation, vibration, human power, andair flow, can be used to solve a problem with long lifetime, asa recharger means to provide power for a long period of timewithout requiring the replacement of batteries.In modern days,the increasing demand of smart autonomous sensor nodes in theInternet of Things applications (like temperature monitoring ofan industrial plant over the internet, smart home automation,and smart cities) requires a detailed literature survey of stateof the art in solar energy harvesting WSN (SEH-WSN) forresearchers and design engineers.
II. WSN COMPONENTS
Conventional Wireless Sensor Networks (WSNs) have thedesign limitation of high power consumption during theiroperation, which has been tackled by mainly duty cycle basedapproaches until now.
Developing of power supply solution, its working princi-ples, and the system architecture is constructed in order tomaximized efficiency. Technology concept and/or applicationformulated (TRL2 complete) show, that prototype of systemis ready to be analytically and experimentally tested in labo-ratory environment critical function and characteristic proof-of-concept. For example, an area of strip is shown in Figure1(a) which is equivalent to the drawn power by WSN nodeswhich transmit data to the adjacent Router. It is to be notedthat WSN node consumes significantly high current, while itremains in active mode. But inside active mode, WSN nodedraws maximum current (e.g., 26 mA) during transmission,while the node consumes significantly different amount ofcurrent for the rest of the active mode operation period asshown in Figure 1(a). From Figure 1(a), it is clear that, byplotting the value of consumed current over a period of timeand summation of the values, an area of strip equivalent tothe amount of power/energy consumed by the WSN node canbe determined. Now to measure the value under the strip, themethodology used. According to this methodology, to measurethe area, an arbitrary continuous function f(x) is used, whichresides in a close interval of a to b The function graph is shownin Figure 1(b).
III. AUTONOMOUS WSN
Solar energy source and chemical energy source are com-bined in a hybrid energy harvester, the components become
257
Figure 1. (a) Example of current consumption of a WSN node fortransmitting a single data pulse to the adjacent WSN Router. (b)Arbitrary continuous functions for a close interval of a to b. [6]
solar and chemical energy sources, dual input harvesting cir-cuitry, rechargeable storage, step-down output circuitry, andWSN components such as node or router as shown in Figure2.
Figure 2. Model of a hybrid energy harvester-based complete WSNsystem
The intermediate energy storage is required, which can storeleftover harvested energy and provide continuous power supplyto the WSN node/router even when there is no ambient energyavailable. Now, to support WSN node/router from the storage,the storage should have capabilities to supply higher voltagethan the adjacent node/router requirement. Output circuitrycomes into action in this scenario by converting the DC/DCdownstream conversion of output power from the storage, ac-cording to the requirement of adjacent node/ router. The overallbehavior of the hybrid powered energy harvester depends onthe design of each component of the system.Solar energyharvesting that provides an alternative power source for anenergyconstrained wireless sensor network (WSN) node. Thesmall size solar panels suitably connected to low-power energyharvester circuits and rechargeable batteries provide a loom tomake the WSN nodes completely self-powered with an infinite
network lifetime. Voltage change steps determine the numberof microcontroller bits and solar cell maximum voltage:
h =UmaxPV
2n, (1)
where h – voltage change step, V; UmaxPV – Maximumvoltage of solar cell, V; n – the number of microcontrollerbits.
Battery charge monitoring is monitored at the same time withthe maximum point detection a level. If the voltage in the loadis higher than the maximum allowed (defined by the user), itwill charge the process is stopped. If the battery voltage falls tothe minimum value, it will charge the process is restored again.This technology, in comparison with analogous solutions, has aclearly defined advantage for use in both sectors of the economyand can be objectively and quantifiably assessed.
The solar energy harvesting simulators use the energy modelof a solar-powered WSN node which is composed of three sub-models: (1) Energy harvesting model, (2) energy consumingmodel, and (3) the remaining battery energy model. The energyharvesting system of a sensor node gathers solar energy using asolar panel that stores the harvested energy in the rechargeablebattery and operates the sensor system using the stored energy.
Solar photovoltaic (PV) energy harvesting refers to con-verting solar light energy into electrical energy to operate anelectrical or electronic device. As applied to WSNs, solar lightenergy is converted into electrical energy and is utilized torecharge the battery of a WSN node at the operation site itself.The electrical energy harvested from solar energy (sunlight)can also be used directly to power a WSN node.
The maximum distance of a Zigbee wireless sensor networkis 100 m, and it can be extended up to 1.5 km. The maximumdata rate of information in the ZigBee protocol is 250 kbpsonly. In figure 3, the basic version of the WSN setup is shown,which can be extended to SEH-WSN by connecting small sizesolar panels.
Figure 3. An SEH-WSN scenario for temperature monitoring applica-tion
IV. ALTERNATIVE POWER SOURCE FOR WSN NODESIN PARTICULAR IRRADIATION CONDITIONS
To measure the quality of the solar cell the Fill Factor (FF)is essential. It is calculated by comparing the maximum power(PMAX) to the theoretical power (PT) that would be output atboth the open circuit voltage and short circuit current together[7].
h =PMAX
PT=VMP IMP
VOCISC, (2)
where VMP – the maximum power point voltage; IMP – themaximum power point current; V OC – the cell open-circuitvoltage; ISC – the cell short-circuits current.
Comparison of FF of the panels does not give us clearindication which type of the tested panels is better for operatingin low irradiation conditions, therefore the Efficiency indicatorn is evaluated. Efficiency is the ratio of the electrical poweroutput Pout, compared to the solar power input, Pin, into the
258
PV cell. Pout can be taken to be PMAX since the solar cellcan be operated up to its maximum power output to get themaximum efficiency.
η =Pout
Pin= ηmax
PMAX
Pin(3)
Pin is taken as the product of the irradiance of the incidentlight, measured in W/m2 or in suns (1000 W/m2), with thesurface area of the solar cell [m2]. The maximum efficiency(nMAX) found from a light test is not only an indication of theperformance of the device under test, but can also be affectedby ambient conditions such as temperature and the intensityand spectrum of the incident light [7].
Due to the test ensured similar lighting and temperatureconditions for PV, the cells can be used for evaluation of thepanel efficiency. Because Pin is the same for all panels, Poutindicates which panels are most efficient for operating in lowirradiation conditions. Actually, A and E panels appeared asmore efficient for the conditions set in the test.
A. Maximum Power Point Tracker for WSN nodes
Maximum Power Point Tracker (MPPT), a power electronicmodule that significantly increases the system efficiency use amicrocontroller based charge controller connected to a batteryand the load is developed in this research. It ensures thecharging of a battery and providing power to the connectedequipment by optimizing the solar panel maximum possiblepower rate at varying lightning conditions [8].
Solar Energy harvesting (SEH) is a technique that scavengesunused ambient solar energy and converts the collected lightenergy into electrical energy. This electrical energy can bestored for future use by the sensor node. The following arethe Design Challenges at the Solar energy harvesting level:
• All the light energy coming from sunlight rays shouldbe fully utilized. The SEH-WSN node should use solarenergy as the primary source and rechargeable batteryenergy as the secondary source.
• To expand the battery charging-discharging life cycle.• Designing of a simple and inventive solar charger.• To shrink the overall power use.• To enhance the stability of the overall SEH-WSN system.• Energy harvester circuits should be compatible exist-
ing WSN industry communication standards like IEEE802.15.4 (ZigBee) and IEEE 1451.5 standards.
• To achieve the highest power from the sun.• To ensure small power consumption for DC-DC Boost
converter operation.• To convey maximum power to the SEH-WSN node using
the harvested energy.• To start-up (or bootstrap) the SEH-WSN node.
• Variations at the solar radiation level, Solar Cell efficiency(n), DC-DC converter design, and MPPT design andEnergy Prediction Algorithms.
• Cost Effective energy harvesting solutions (cheaper thanthe battery replacement cost).
Pertaining to the use of solar energy to power WSNs, alot of researchers have done a lot of research work. But still,there are many design challenges in SEH-WSNs, which needto be explored for further optimization. The design challengesin SEH-WSN are shown in Fig. 4.
The internal block diagram of an WSN node is displayedin figure 5. An WSN node consists of the following two mainunits [11]:
Figure 4. Design challenges in SEH-WSN
Figure 5. WSN node connected to solar panels
B. MicrocontrollerThe ATTINY13A [9] task is to combine the solar panel
voltage measurements with ADC, to drive the digital poten-tiometer by ensuring the optimal charging current selection andto control the load disconnection and connection. The interfaceof the digital potentiometer is SPI where the pin number 8 ofthe microcontroller is used.
C. Microcontroller software – control algorithmThe control software is modular: ADC, digital potentiometer
driver, disconnection module and algorithm logical machine.ADC measured input voltage of the solar panel, for the selectedpanel it is in the range up to 20V. The divisor reduces thevoltage to a reference voltage and the software finds thecurrent voltage value. This voltage is the starting point forthe determination of the optimal power point for the selectedsolar panel. This point must be observed by monitoring thenominal voltage drops and adapting of the charging current[10]. Here is a simple application of the research results. Theaverage current of a gateway developed at Beagle Bone boardis 350mA. Imagine that for particular WSN it is sufficient, ifa gateway operates at least 8 hours per one twenty-four hours.Therefore, 350 mA x 5V x 8 hours = 14000 mAh. The bestsolar plate, tested in the paper, theoretically has Pout about9000 mW/m2 that is 9000 mW/m2 x 0.25 m2 = 2250 mW – thePout of the one panel. So 14000 mWh: 2250 mW = 6.2 hours.It means that for our case one solar panel is able to ensurefeeding of the device in the summer during 8 hours, but in thewinter, when the day light is much shorter and is less solardays, one solar panel is not enough. To ensure the operation
259
of a particular WSN, the decision can be made to use a lowerpower rating solution like a repeater node to compensate forthe lower solar irradiation levels, in any case the evaluation ofeach individually deployed PV panel is vital to ensure criticaldata transmission. However, for reliable result for real WSN itis necessary to make also field research on solar irradiation indifferent time of the day and time of the year.
CONCLUSIONS
This paper makes the following contributions in the fieldof autonomous WSN. The fundamental concepts of an au-tonomous WSN system have been explained to understandthe state of the art and operation of an autonomous WSN.We survey the autonomous WSNs at four basic levels, i.e.,energy harvesting, sensing, computation, and communicationlevels with various design challenges.
In this paper, a thorough investigation has been carried out todesign and implement a hybrid energy harvester that is capableof harvesting energy from ambient sources to powerup theWSN nodes. The authors of the research offered an approachthat can help to use PV panels as alternative power sourcefor WSN nodes in particular irradiation conditions. Surveyand testing of the main types of PV panels offered in themarket in conditions closed to real ones, in which WSN nodesare maintained, was implemented. Based on test results, amaximum power control module parameter can be calculatedin order to achieve the best effectiveness of the power controlsystem. The conclusions were made that conditions one solarpanel is able to ensure feeding of the device in the summerduring particular hours, but in the winter, when the day lightis much shorter and is less solar days, one solar panel is notenough. The novelty of the research is PV testing method andselection of design and MPP control module parameters, whichensure maximum effectiveness of WSN nodes power feeding.
The future scope of autonomous WSN is very promising inthe field of smart homes using sensors (Home Automation),smart buildings, smart cities, environment monitoring, indus-trial process control, security, and the Internet of Things (IoT)applications.
We draw attention to the fact that one of the possible use ofthis research may be the recognition and control of speech onthe basis of various kinds of sensors.
REFERENCES
[1] V. J. Hodge, S. O’Keefe, M. Weeks, A. Moulds, "Wireless sensornetworks for condition monitoring in the railway industry: asurvey", IEEE Trans. Intelligent transportation systems, pp. 1-19,2014.
[2] M. Grisostomi, L. Ciabattoni, M. Prist, G. Ippoliti, S.Longhi,"Application of a wireless sensor networks and Web2Pyarchitecture for factory line production monitoring", 11th IEEEInternational multi-conference on systems signal & devices(SSD), pp. 1-6, 2014.
[3] C. Popeanga, R. Dobrescu, N. Cristov, "Smart monitoring andcontrolling of wind turbines farms based on wireless sensorsnetworks", 1st International conference on System and ComputerSience (ICSCS), pp. 1-6, 2012.
[4] Ch. Ranhotigamage, S. Chandra Mukhopadhyay, "Field trials andperformance monitoring of distributed solar panels using a low-cost wireless sensors network for domestic applications", IEEEJ. Sensors, vol. 11, no. 10, pp. 2583-2590, 2011.
[5] Ribickis, L., Kun, icina, N., Zabašta, A., Galkina, A., Caiko, J.,Kondratjevs, K.,etc. Sensor Network Technology Applications inthe Water Supply and Transport Systems. Riga: RTU, 2017. 194p. ISBN 978-9934-10-915-7.
[6] Protege. Available at https://www.hindawi.com/journals/ijp/2014/760534/ (accessed 2019, Jan)
[7] Part II – Photovoltaic Cell I-V Characterization Theoryand LabVIEW Analysis Code (2012) http://www.ni.com/white-paper/7230/en/
[8] A. Frezzetti, S. Manfredi, M. Pagano, "An implementation of asmart maximum power point tracking controller to harvest renew-able energy of wireless sensor nodes",International Conference onClean Electrical Power (ICCEP) , Page(s):503- 508, 2013.
[9] Protege. Available at https://www.microchip.com/wwwproducts/en/ATtiny13A (accessed 2019, Jan)
[10] Stuart Ball. Analog Interfacing to Embedded Microprocessors.Newnes, 2001.
[11] Protege. Available at https://www.researchgate.net/publication/323731881 (accessed 2019, Jan)
ПОТРЕБЛЕНИЕ ЭНЕРГИИ АВТОНОМНЫМИУЗЛАМИ В БЕСПРОВОДНЫХ СЕНСОРНЫХ
СЕТЯХ
Чайко Е. В., Витулёва Е. С., Солощенко А. В.
Параметр внешней среды, использующий большоеколичество пространственно-распределенных узловбеспроводной сенсорной сети (WSN), является обшир-ной иллюстрацией передовых современных техноло-гий, но высокие требования к мощности для узловWSN ограничивают широкое распространение этихтехнологий. В настоящее время узлы WSN активнопитаются от батарей, но у батареи есть ограничения посроку службы, плотности мощности и экологическимпроблемам. В последнее время беспроводные сенсор-ные сети (WSN) широко используются в автоматиза-ции. Источник питания WSN оказывает значительноевлияние на их производительность. В этом документепредставлена новая схема управления питанием дляWSN.
Предлагаемый узел должен иметь возможность от-правлять уведомления в программу, требуя использо-вания резервных энергетических стратегий.
Авторы исследования предложили подход, которыйможет помочь использовать накопитель EH на основесолнечной энергии в качестве наиболее жизнеспособ-ного источника энергии для автономных узлов WSN.По результатам испытаний разработаны параметрывстроенного модуля управления мощностью. Новизнаисследования — это подход, который включает в себяразличные решения в области источников питания,чтобы обеспечить определенные параметры качестваперехода сигнала для максимальной эффективностиподачи энергии узлами WSN.
Авторы обращают Ваше внимание на тот факт, чтоодним из возможных применений этого исследованияможет быть распознавание и контроль речи на основеразличных видов датчиков.
Ключевые слова: беспроводные сенсорные сети,WSN, солнечная энергия, сбор энергии.
Received 10.01.19
260
Sensor Location Problem’s SoftwareOptimization
Andrei PilipchukBelarusian State University
Minsk, [email protected]
Ludmila PilipchukBelarusian State University
Minsk, [email protected]
Eugene PolyachokBelarusian State University
Minsk, [email protected]
Abstract—In this work we consider the application of thegraph theory for construction the optimal and suboptimalsolutions to the sensor location problem. That problemis named Sensor Location Problem for a graph (SLP).For constructing the solution of the SLP for a graph wepresented the pseudocodes of the algorithm’s for finding theflow arcs for the non observered part on the network. In thepseudocode of the algorithm 1 defines sensor configurationsof the suboptimal solution and flows on the arcs on theunobserved part of the network.
Keywords—software optimization, sensor location prob-lem, sparse linear system, suboptimal and optimal solution
I. INTRODUCTION
The problem of locating sensors on the network tomonitoring flows has been object of growing interest inthe past years, due to its relevance in the field of trafficmanagement and control [1]–[4]. The basis for modelingthe processes of estimating flows in network is a sparseunderdetermined systems of linear algebraic equations ofa special types [5], [6]. Sensors are located in the nodesof the network for the given traffic levels on arcs withinrange covered by the sensors, that would permit trafficon any unobserved flows on arcs to be exactly.
This work is devoted to the research of intelligenttransport systems and their applications. The obtainedtheoretical and practical results are an important contri-bution to the solution of problems in the field of environ-mental monitoring. Technologies and algorithms for onepractical solution problem of ecological monitoring andanalysis of flows on the unobserved part of the transportnetwork are developed.
The suboptimal solutions for the network program-ming problem are considered in [7]. The common solu-tions for the sparse underdetermined systems of linearalgebraic equations are obtained in [8]. In this workwe research the numerical results for constructing thesuboptimal solutions of SLP problem for various valuesof the intensity threshold.
II. SENSOR LOCATION PROBLEM
Let’s introduce the finite connected directed graphG = (I, U). The set U is defined on I × I (|I| <
∞, |U | < ∞). We assume, that the graph G is sym-metric: that is: if (i, j) ∈ U , then (j, i) ∈ U. We notethat the graph G is not undirected: the flow on arc (i, j),in general, will not be the same as the flow on arc(j, i). To designate this distinction, we refer to the graphG = (I, U) as a two way directed graph.
We represent the traffic flow by a network flow func-tion x : U → R that satisfies the following system:
∑
j∈I+i (U)
xi,j −∑
j∈I−i (U)
xj,i =
xi, i ∈ I∗,0, i ∈ I \ I∗ (1)
where I∗ is the set of nodes with variable intensities, xiis the variable intensity of node i ∈ I∗, I+i (U) = j ∈I : (i, j) ∈ U and I−i (U) = j ∈ I : (j, i) ∈ U. Ifthe variable intensity xi of node i is positive, the nodei is a source; if it is negative, this node i is a sink. Forsystem (1) is true the following condition:
∑
i∈Ixi = 0.
According [9] if I∗ 6= ∅, then the rank of the matrix ofsystem (1) for a connected graph G = (I, U) is equalto |I|.
In order to obtain information about the network flowfunction x and variables xi of nodes i ∈ I∗, sensorsare placed at the nodes of the graph G = (I, U). Thenodes in the graph G = (I, U) with sensors we callmonitored ones and denote the set of monitored nodesby M, M ⊆ I . We assume that if a node i is monitored,we know the values of flows on all outgoing and allincoming arcs for the node i ∈M :
xij = fij , j ∈ I+i (U), xji = fji, j ∈ I−i (U), i ∈M.
If the set M includes the nodes from the set I∗, then wealso know the values xi = fi, i ∈M
⋂I∗. So, we have
xij = fij , j ∈ I+i (U), xji = fji, j ∈ I−i (U),i ∈M ;
xi = fi, i ∈M⋂I∗.
(2)
Consider any node i of the network. For every out-going arc (i, j) ∈ U for this node i determine a realnumber pij ∈ (0, 1] which denotes the part of the total
261
outgoing flow∑
j∈I+i (U)
xij from node i corresponding to
the arc (i, j). That is,
xij = pij∑
j∈I+i (U)
xij .
If |I+i (U)| ≥ 2 for the node i ∈ I then we can writethe arc flow along all outgoing arcs from node i (exceptany selected arc) in terms of arc flow a single outgoingarc, for example, (i, vi), vi ∈ I+i (U):
xi,j =pi,jpi,vi
xi,vi , j ∈ I+i (U) \ vi. (3)
We continue this process for each node i ∈ I, if|I+i (U)| ≥ 2.
Let |I+i (U)| ≥ 2 for any node i ∈ I and xi,viis known
for the arc (i, vi) and equal to fi,vi. Then we can write
the unknown arc flow along all outgoing arcs from nodei (except any selected arc (i, vi)) in terms of arc flow fora single outgoing arc (i, vi), where xi,vi is known andequal to fi,vi
.Let’s substitute the calculated arc flows according to
(2) and (3) in the equations of system (1). Let’s deletefrom graph G = (I, U) the set of the arcs on which thearc flow are known. Let’s delete from graph G the setof the nodes i ∈ M . Then we have a new graph G =(I, U). A new set of nodes with variable intensity for anew graph G is I
∗, where I
∗= I∗\(M ⋂
I∗). The newgraph G can be non-connected. The graph G consistsof connected components. Some connected componentsmay contain no nodes of the set I
∗. The system (1) for
graph G = (I, U) will be the following one:
∑
j∈I+i (U)
xi,j −∑
j∈I−i (U)
xj,i =
xi + bi, i ∈ I∗,ai, i ∈ I \ I∗
(4)∑
(i,j)∈Uλpijxij = 0, p = 1, q, (5)
where ai, bi, λpij – are constants.
So formulate the optimal solution to the Sensor Loca-tion Problem: what is the minimum number of monitorednodes |M | such that system (4)–(5) has an uniquesolution?
In [10] was proof that SLP problem is NP-complete.
III. OPTIMAL SOLUTION TO THE SENSOR LOCATIONPROBLEM
In Figure 1 we show a finite connected directedsymmetric graph G with the set of nodes I and the setof arcs U where
I = 1, 2, 3, 4, 5, 6,U = (1, 2), (1, 3), (2, 1), (2, 4), (2, 6), (3, 1), (3, 5), (4, 2),
(4, 6), (4, 5), (5, 3), (5, 4), (5, 6), (6, 2), (6, 4), (6, 5),
I∗ = 2, 4, 5, 6.
Figure 1. Finite connected directed symmetric graph G
For the graph G = (I, U) (see Figure 1) we write thesystem of linear algebraic equations in the form:
x1,2 + x1,3 − x2,1 − x3,1 = 0
x2,4 + x2,6 + x2,1 − x4,2 − x1,2 − x6,2 = x2
x3,1 + x3,5 − x1,3 − x5,3 = 0
x4,2 + x4,5 + x4,6 − x2,4 − x5,4 − x6,4 = x4
x5,4 + x5,3 + x5,6 − x3,5 − x4,5 − x6,5 = x5
x6,2 + x6,4 + x6,5 − x2,6 − x4,6 − x5,6 = x6
(6)
Suppose that the set of monitoring nodes is M = 2for the graph shown in Figure 1. Construct the cutCC(M) with respect to the set M . We form the sets
M+ = I(CC(M)) \M = 1, 4, 6;
M∗ =M⋃M+ = 1, 2, 4, 6;
I \M∗ = 3, 5.
In the Sensor Location Problem (SLP) the values offlows on all incoming and outgoing arcs for the eachnode i of the set M (monitored nodes) are known andwe also know the values xi = fi, i ∈M
⋂I∗ :
x1,2 = f1,2, x2,1 = f2,1, x2,4 = f2,4,
x4,2 = f4,2, x2,6 = f2,6, x6,2 = f6,2, x2 = f2.(7)
We substitute the known values of the variables (7) tothe system of equations (6) and delete the correspondingarcs from the graph G. Also, we delete the nodes i ∈Mfrom the graph G. The graph G′ obtained after deletingthe arcs corresponding to the variables (7) and nodesi ∈ M from graph G is shown in Figure 2. The rest ofthe flows for the outgoing arcs from the nodes of the setM+ = I(CC(M)) \M = 1, 4, 6, can be expressedfrom the flows of the outgoing arcs for M+ = 1, 4, 6by the following equations:
262
x1,3 =p1,3p1,2
f1,2, x4,5 =p4,5p4,2
f4,2,
x4,6 =p4,6p4,2
f4,2,
x6,4 =p6,4p6,2
f6,2, x6,5 =p6,5p6,2
f6,2.
(8)
Let us substitute (8) to the system of linear equations(6). We delete from the graph G arcs which correspond tothe known values of the arc flows (7) and (8). The graphG = (I, U) obtained by deleting the arcs correspondingto variables (8) from the graph G′ is shown in Figure 3.The system (6) for the graph G = (I, U) (see Figure 3)transforms to the form (9).
Figure 2. Graph G′
f1,2 +p1,3p1,2
f1,2 − f2,1 − x3,1 = 0,
f2,1 + f2,4 + f2,6 − f1,2 − f4,2 − f6,2 = f2,
x3,1 + x3,5 −p1,3p1,2
f1,2 − x5,3 = 0,
f4,2 +p4,5p4,2
f4,2 +p4,6p4,2
f4,2−
−f2,4 − x5,4 −p6,4p6,2
f6,2 = x4,
x5,4 + x5,3 + x5,6 − x3,5 −p4,5p4,2
f4,2−
−p6,5p6,2
f6,2 = x5,
f6,2 +p6,4p6,2
f6,2 +p6,5p6,2
f6,2−
−f2,6 −p4,6p4,2
f4,2 − x5,6 = x6.
(9)
Arc flows xi,j , (i, j) ∈ U , corresponding to the arcsoutgoing from node set I \M∗ = 3, 5 are unknown.For these unknown flows xi,j , (i, j) ∈ U we form theadditional equations.• Choose arbitrary outgoing arc that starts from a
node set i of set I \ M∗ = 3, 5, for example,for the node i = 3 we choose the arc (3, 5). Let usexpress the arc flows to all other arcs outgoing from
Figure 3. Graph G = (I, U)
the node i = 3 through the arc flow of x3,5 for thechosen outgoing arc (3, 5).
• Choose any outgoing arc from the node i = 5, forexample (5, 6). Let us express the arc flows to allother arcs outgoing from the node i = 5 throughthe arc flow of x5,6.
x3,1 =p3,1p3,5
x3,5, x5,3 =p5,3p5,6
x5,6, x5,4 =p5,4p5,6
x5,6.
Additional equations have the form:
x3,1 −p3,1p3,5
x3,5 = 0, x5,3 −p5,3p5,6
x5,6 = 0,
x5,4 −p5,4p5,6
x5,6 = 0.(10)
Part of the unknowns of the system (9), (10) makes upoutgoing arc flows for arcs from node sets I \ M∗ =3, 5 for the graph G :
x3,1, x3,5, x5,3, x5,4, x5,6.
The remaining part of the unknowns of the system (9),(10) defines the variables xi, i ∈ I
∗= 4, 5, 6:
x4, x5, x6.
Thus, the system (9), (10) is a system of full rank. Thenumber of unknowns of the system (9), (10) equal to therank of the matrix and is equal to 8. The system (9), (10)has the unique solution for given set of monitored nodesM = 2.
IV. PSEUDOCODE ALGORITHMS SUBOPTIMALSOLUTION TO THE SENSOR LOCATION PROBLEM
In the work [11] get the interval [1, |I∗|] valueschanges of the number of |M | nodes being viewed. Sub-optimal (t−optimal) solution are constructed to the SLPproblem of the establishment of full observability of thenetwork for given intensity threshold t: |xi| ≥ t, i ∈ I∗.We presented the pseudocode algorithm’s for finding thesuboptimal solution to the sensor locations problem.
263
Algorithm 1 Pseudocode algorithm’s for the suboptimalsolution to the sensor locations problemList of reference symbols:arcs - array arc with flows valueFAKE-VERTEX - fake vertexflow - value flow arciterations - countlowerBound - value lower boundnormArcs - array arc notmalize with flownumberThreads - count children threadsnumberTrials - count valueobservedArcs - array observible arcsthreshold - threshold of intensity
Input: arcs, threshold, iterations, numberTrials,lowerBound, numberThreads
Output: sensors configuration1: normArcs← balance_nodes(arcs, treshold) . Algorithm
22: initialV ertices← get_start_nodes(normArcs) .
Algorithm 33: if FAKE − V ERTEX ∈ initialV ertices then4: delete FAKE-VERTEX from initialVertices5: end if6: randomSearch ← rsls(normArcs, initialVertices, itera-
tions, numberTrials, lowerBound, numberThreads) .Algorithm 4
7: observedV ertices← null8: for all results ∈ randomSearch do9: if residual ∈ result < 10−10 and cond ∈ result <
106 then10: observedV ertices← vertices ∈ result11: break12: end if13: end for14: observedArcs ← get_monitored_nodes_with_network
(normArcs, observedVertices) . Algorithm 515: get_sensors_configuration(normArcs, observedVertices,
observedArcs) . Algorithm 6
Algorithm 2 – pseudocode of the algorithm to obtain-ing an array of normalized arcs.
Algorithm 3 – pseudocode of the algorithm to obtain-ing an array of initial nodes with variable intensity.
Algorithm 4 – pseudocode of the algorithm of therandom search location sensors.
Algorithm 5 – pseudocode of the algorithm for obtain-ing the array of observed arcs.
Algorithm 6 – pseudocode of the algorithm to ob-taining of the sensor configurations for the suboptimalsolutions.
V. SUBOPTIMAL SOLUTION TO THE SENSORLOCATION PROBLEM
The example of a suboptimal solution show in thefigure 4 for the graph G = (I, U), |I| = 9, |U | = 18I∗ = 2, 4, 5, 6, 7, 9.
After location in the network G the |M | = 6 sencorsin the nodes M = 2, 4, 5, 6, 7, 9, (fig. 4), we have thesuboptimal solution.
Figure 4. Suboptimal solution: M = 2, 4, 5, 6, 7, 9
REFERENCES
[1] Pilipchuk L.A , Pilipchuk A.S., Pesheva Y.H. Graph Algorithmsin Sparse Linear Systems with Rectangular Matrices. AmericanInstitute of Physics (AIP), 1570, 2013, pp. 485–490.
[2] Bianco L., Confessore G., Gentili M. Combinatorial Aspects ofthe Sensor Location Problem. Annals of Operation Research,144 (1), 2006, pp. 201 – 234.
[3] Pilipchuk L. A. Malakhouskaya Y. V., Kincaid D. R. andLai M. Algorithms of solving large sparse underdeterminedlinear systems with embedded network structure. East-West J.of Mathematics. Vol. 4, No. 2, 2002, pp. 191–201.
[4] Bianco L., Cerrone , Cerulli R., Gentili M. Locating Sensors toObserve Network Arc Flows: Exact and Heuristic Approaches.Computers and Operation Research. Vol. 46, 2014, pp. 12 – 22.
[5] Pilipchuk L.A, Pilipchuk A.S. Sparse linear systems: theory of de-composition, methods, technology, applications and implementa-tion in Wolfram Mathematica. American Institute of Physics. AIPConf. Proc. Vol. 1690, 060006 (2015); doi: 10.1063/1.4936744.– 9 p.
[6] Pilipchuk L. A., Vishnevetskaya T. S., Pesheva Y. H. Sensorlocation problem for a multigraph. Math. Balk. New Ser. Vol.27, Fasc. 1–2, 2013, pp. 65–75.
[7] Pilipchuk L., Pilipchuk A., Pesheva Y. Algorithms for construc-tion of optimal and suboptimal solutions in network optimizationproblems. International Journal of Pure and Applied Mathemat-ics. Vol. 54, No. 2, 2009, pp. 193 – 205.
[8] Pilipchuk L. A., German O. V., Pilipchuk A. S. The generalsolutions of sparse systems with rectangular matrices in theproblem sensors optimal location in the nodes of a generalizedgraph. Vestnik BGU. Ser. 1, Fiz. Mat. Inform. 2015, No. 2, pp.91– 96.
[9] Pilipchuk L.A. Sparse Linear Systems and Their Applications –Minsk : BSU, 2013.
[10] Bianco L., Confessore G., Gentili M. A network based model fortraffic sensor location with implication in O/D matrix estimates.Transportation Science. Vol. 35, No. 1, 2001, pp. 50 – 60.
[11] Pilipchuk, L.A., Pilipchuk A.S., Polyachok E.N., Farazei A.I.Identifikatsiya sensornoi konfiguratsii i upravlenie potokami.Zhurnal Belorusskogo Gosudarstvennogo Universiteta. Matem-atika. Informatika. – No 2, 2018, pp. 67–76.
ОПТИМИЗАЦИЯ ПРОГРАММНОГООБЕСПЕЧЕНИЯ ПРОБЛЕМЫРАСПОЛОЖЕНИЯ СЕНСОРОВ
Пилипчук А., Пилипчук Л., Полячок Е.В работе рассматривается приложение теории графов для
построения оптимальных и субоптимальных решений задачирасположения сенсоров. Эта задача называется проблемойрасположения сенсоров (SLP) для графа. Для построениярешений задачи SLP для графа мы представляем псевдокодыалгоритмов для нахождения дуговых потоков на ненаблюда-емой части сети. В псевдокоде алгоритма 1 определяютсясенсорные конфигурации субоптимального решения и дуго-вые потоки на ненаблюдаемой части сети.
Received 29.12.18264
Algorithm for Fast Image Compression onHeterogeneous Computing Devices
Vadim V. Matskevich, Viktor.V. KrasnoproshinBelarusian State University
Minsk, [email protected], [email protected]
Abstract—In spite of the intensive development of com-puter technology, the problem of efficient use of computingresources remains an urgent issue [1], [2]. Computingdevices may be heterogeneous (different in architectureand power). In this case, when solving applied problems,it becomes necessary to efficiently load them [3]–[6].
The paper proposes an algorithm for loading heteroge-neous devices, which allows speeding up the data processingprocess when solving the problem of image compression.
Keywords—Parallel computing, computing devices, sys-tem performance, neural network, training, dataset.
I. IMAGE COMPRESSION USING NEURAL NETWORK(NN)
In the general case, a color raster image is described bya vector, each coordinate of which reflects its particularcharacteristic. Coordinate values can, for example, define[6]:
– the number of pixels in the image;– color wavelength values;– coordinates of specific pixels (if the image has an
arbitrary shape);– and etc.Storage of color raster images in the form of a vector
requires, as a rule, large amounts of memory [7]. Thereis a problem of their reduction, i.e. It is necessary tosolve the problem of image compression.
Consider the process of compressing color imagesusing a forward passing neural network (NN) [7] (seefigure 1).
In general, the process of compressing images usingneural networks consists of the following main steps.
Stage 1. The format of the original image is deter-mined. Usually the image is considered as raster.
Stage 2. Vector (x1, x2, ..., xn), describing the originalimage is formed. For color images, each pixel is usuallydescribed by three coordinates of the vector. One coor-dinate determines the content of red, the second – thegreen and the third - the blue.
Stage 3. The neural network configuration is selectedto implement the image compression process. The num-ber of neurons in the input layer always corresponds tothe number of coordinates in the vector (x1, x2, ..., xn).The output layer determines the size of the output vector(y1, y2, ..., ym).
Stage 4. As a result of neural network data processing,a vector is constructed (y1, y2, ..., ym), which describesthe original image already in a compressed form (m ≤n).
When solving the problem, the NN architecture of thefollowing configuration was used.
The sizes of the input and output layers consisted of48x48x3 and 12x12 neurons, respectively. Neighboringlayers were represented as full bipartite graphs. In allneurons (except the input layer), a bipolar sigmoid func-tion of activation with a unit coefficient was used.
f(S) =2
1 + e−cS− 1 (1)
The initial weights of the NA were generated uni-formly distributed over the interval of values
[− 16√
l1,16√l1
](2)
where l1 – is the number of neurons in the firstlayer (when learning, the output signals of this layer areconsidered as input data for training).
Using the same formula, the values of the fictitiousautoencoder weights were generated.
The learning rate (lr) for each layer throughout thetraining was a constant value and was calculated by theformula:
lr =2√l1
(3)
In contrast to the usual multilayer perceptron, theconstructed network was trained as a deep NN.
II. ALGORITHM OF LOADING HETEROGENEOUSCOMPUTING DEVICES
Color images were represented by a set of independentcopies, formed in a separate package S. For compression,n computational (heterogeneous) Ui, i = 1, n powerdevices were used.
It was necessary to construct such a partition of S inton parts, which would minimize the processing time ofthe entire package.
We denote by Si the set of images processed on thei-th device and Ti – the time of its processing. Then,
265
Figure 1. Image compressing
formally, the partitioning problem can be written asfollows:
Si ∩ Sj = ∅⋃i=1,n
Si = S
maxi=1,n
Ti → min
(4)
Note. In the problem, it is important to find the valuesof |Si| that satisfy (4).
Denote by P – the amount of computation requiredto process a single image from S. It is known that theprocessing time of input signals in forward propagationNN is constant.
To achieve the minimum processing time, two condi-tions must be met. The computing devices must work (a)simultaneously and (b) the same amount of time.
With this in mind, the task can be rewritten as:
P |Si|Ui
= P |Sn|Un
, ∀i = 1, n− 1n∑
i=1
|Si| = |S|(5)
in the formula, the restriction on the coverage of allimages of the package is omitted, because it is obvious.Fix the parameters Ui, i = 1, n, and solve the system(5):
|Sn| = |S|n−1∑i=1
UiUn
+1
|Si| = Ui
Un|Sn| , ∀i = 1, n− 1
(6)
When calculating |Si|, ∀i = 1, n unknown in (6)remain the ratio of power. There are two ways to getthem:
1) you can use the table values specified by themanufacturer. This is not suitable in the case ofvideo cards. The same video card, depending onthe specifics of the task, may have different power;
2) it is possible to experimentally calculate the powerratio of computing devices.
The following method for estimating power is pro-posed.
Let x be the subset of images needed to process thepackage S, and τi is the i-th device operation time onthis subset. Then we get the system of equations:
τi =P |x|Ui
,∀i = 1, n (7)
It is not difficult to notice that the time of the workof computing device at a fixed number of operations isinversely proportional to its power.
In order to obtain the power ratio, the last (n-th)equation we divide into n− 1 previous ones.
Ui
Un=τnτi,∀i = 1, n− 1 (8)
The expression (8) means that the power ratio isinversely proportional to the ratio of time spent onprocessing a fixed number of operations.
The resulting expression accurately reflects the powerratio. It takes into account the actual conditions in whichthe calculations are made.
An approach to the efficient use of resources based ondata parallelization technology is proposed. Formally, itcan be described as the following algorithm.
Step 1. From the set of images S select (small inpower) a subset necessary for processing the batch S.
Step 2. Process this subset (with time measurement fortheir execution) on each of the devices.
Step 3. According to the formula (8) determine thepower ratio.
Step 4. By the formula (6) calculate the optimal loadfor each computing device. For this:
– the set of images S is divided into |S| subsets ofPj , j = 1, |S| ,
– for each i-th device i = 1, n, a subset of Si isformed by join Pj , i.e. we have:
Si =⋃
h=1,|Si| Pjih⋃i=1,n
Si = S
Si ∩ Sj = ∅, ∀i 6= j
(9)
266
We assume that the devices are commensurate witheach other in power. This condition is satisfied if:
maxi=1,n
∪i ≤ c minj=1,n
∪j (10)
where c is a constant equal to two.2) The package of images S must be large enough. In
this case, it makes sense to parallelize their processing.3) After calculating by formula (6) some values of
the powers |Si|, i = 1, n may turn out to be fractionalnumbers. In this case, it is necessary to round them toa integer, taking into account the restriction (5). If |Si|,i = 1, n is substantially greater than n, then roundingwill not practically slow down the computation time.Otherwise, it is necessary to round up to an integer(according to standard mathematical rules). In the caseof "shortage", "free" images of the package S must betransferred to computing devices in descending order oftheir power. In the case of "busting" – remove one imagefrom computing devices in order of increasing power.
The proposed approach has several advantages:1) When solving applied problems, one can efficiently
use the computing resources of a standard com-puter. They usually contain a multi-core processorand a video card.
2) High accuracy estimates of the optimal |Si|, i =1, n in (6) is guaranteed.
3) In the process of solving the problem, no additionalcalculations are required, but simply to solve thesystem (6).
III. EXPERIMENTS
The STL-10 data from the Stanford University repos-itory [8] was used as the source data. The datasetcontains one hundred thousand unmarked color imagesmeasuring 96x96 pixels. Each image is described by27648 integers (in the range from 0 to 255) specifyingthe content of red, green, and blue colors [9]. Based onthe obtained characteristics (the sample is set to about2.8 * 109 numbers, contains descriptions of arbitrary,unmarked objects), we can conclude that the process ofcompressing the images of this sample with low lossesis quite a challenge.
To illustrate the nature of the data used in the exper-iment, we give examples of some instances of imagesfrom the STL-10 dataset. (see Figure 2).
For data processing, a standard computer with an 8-core processor and a video card was used:
Video card nvidia – 1050 2gb; processor – amd ryzen7 1700 3.0 GHz; RAM: 2x8 Gb 3200 MHz; Hard drive– samsung 850 pro 256 Gb; operation system – Lubuntu16.04.3.
Computer graphics card is about 60% more powerfulthan the processor. Power of devices are commensu-rate with each other. Therefore, the configuration of
the computing heterogeneous device corresponds to theconditions noted above.
The nvcc 7.5 compiler was used as software (libariesCUDA (Version 9.1 [10]) and OpenMP) with options:
nvcc -D_FORCE_INLINES -O2 –machine 64 -lgomp-Xcompiler -fopenmp program_code.cu
Measurement of time of operations was carried outusing the function "gettimeofday".
Note. The compiler option "–machine 64" shows thatthe application is 64 bit, therefore, it is not compatiblewith 32 bit systems.
When training a neural network, batch mode was used.The batch size was fixed and equal to 6561 images. Thelatter was due to the following considerations:
The size of the training dataset is one hundred thou-sand images. This number is not divisible by three,therefore, the degree of the triple can be chosen as thebatch size to maximize the LCM and, therefore, as therepetition period of the batches. The batch size is threeto the eighth degree, hence the repetition period of thebatches is 656100000 images.
As part of the computational experiment, two differentcases were considered:
– image batch processing is carried out only on avideo card,
– simultaneously on the processor and video card(using the developed approach).
The general scheme of the experiments included thefollowing main steps:
1) Read input data.2) Preliminary image processing.3) Initialization of the initial values of the weights of
the neural network.4) "Endless" learning cycle (each batch includes 6561
images).a) The learning process without a teacher.b) Measuring training time for 15 packets and
checking the conditions for exit from thecycle.
The criterion for exiting the cycle: «the total root-mean-square error on 15 image packets increased». In theprocess of implementing the experiment (in accordancewith the described approach), the following actions wereperformed:
– on the first iteration of the «infinite» cycle on thevideo card, the first half of the image batch isprocessed and the processing time τgpu is measured;
– the second half of the packet is processed on theprocessor and the time τcpu is measured;
– using formulas (8) and (6), the number of imagesthat are fed to the processor and video card, respec-tively, is calculated;
– adjustment of neural network weights is carried out;– the number of images of the batch, which is cal-
culated on the first iteration, is processed on the267
Figure 2. STL-10 pictures examples [9]
processor and video card simultanioulsy (in theloop);
– the neural network weights are adjusted again.According to the results of the experiments, the fol-
lowing results were obtained (see table 1):
Table IPROBLEM SOLVING TIME
Used computing devices Videocard Processorand video cardTimeof processing (in sec.) 24 15
It is easy to see that the use of the algorithm hasreduced the processing time by 60%.
Thus, due to the rational loading of heterogeneouscomputing devices, it is possible to reduce the totalprocessing time. This is important when solving appliedproblems in real-time conditions.
IV. CONCLUSION
The paper presents an algorithm for calculating theloading of heterogeneous devices, which reduced theprocessing time for image compression.
The efficiency of the algorithm is demonstrated on acomputer with a multi-core processor and a video card.
The ideas described in the paper may be useful inprocessing large amounts of data on heterogeneous clus-ter calculators that are being actively developed at thepresent time.
REFERENCES
[1] Pavan Balaji Programming models for parallel computing (Sci-entific and engineering computation) / Pavan Balaji. – The MITpress, 2015 - 488p. ISBN – 978-0262528818
[2] Voevodin, V. V. Parallel computations [Text]: book. / V. V.Voevodin – Piter: BXV Saint Petersberg, 2004. – 608p. – ISBN5-94157-160-7.
[3] Gregory R. Andrews Foundations of multithreaded, parallel anddistributed programming [Text]: book – 688p. – ISBN 978-0201357523
[4] Cavaro-Menard C. Compression of biomedical images and signals/ C. Cavaro-Menard – Wiley, 2013. –288p. – ISBN 978-1-84821-028-8
[5] Burger W. Principles of digital image processing / W. Burger. –Springer, 2013. – 369p. – ISBN 978-1-84882-919-0
[6] Marz N., Warren J. Big Data: Principles and Best Practices ofScalable Real-time Data Systems / N. Marz, J. Warren – ManningPublications, 2015 – 328p. – ISBN 978-1617290343
[7] Simon Haykin Neural Networks and Learining machines (thirdedition) / Simon Haykin. – Pearson Prentice Hall, 2009. – 936p.– ISBN 978-0-13-147139-9
[8] STL-10 dataset [electronic resource]. – link : academictor-rents.com/details/a799a2845ac29a66c07cf74e2a2838b6c5698a6a– Access date: 25.02.2018.
[9] STL-10 dataset description [electronic resource]. – link : stan-ford.edu/ acoates//stl10/ – Access date: 24.02.2018.
[10] CUDA toolkit [electronic resource]: – link :developer.nvidia.com/cuda-downloads – Access date: 23.02.2018.
АЛГОРИТМ БЫСТРОГО СЖАТИЯИЗОБРАЖЕНИЙ НА ГЕТЕРОГЕННЫХВЫЧИСЛИТЕЛЬНЫХ УСТРОЙСТВАХ
Мацкевич В.В., Краснопрошин В.В.В работе рассмотрена проблема организации эффектив-
ной обработки данных на гетерогенных вычислительныхустройствах. Предложен один из возможных подходов крешению проблемы с использованием технологии распарал-леливания данных. Показано, что в общем случае проблемапредставляется нетривиальнойматематической задачей.Дляодного из частных случаев предложен алгоритм решения.Эффективность подхода демонстрируется на примере реше-ния задачи сжатия цветных изображений с использованиемнейронной сети прямого распространения. Описанные вработе идеи могут оказаться полезными при обработкебольших объемов данных на гетерогенных кластерных вы-числителях.
Ключевые слова: Параллельные вычисления, вычисли-тельные устройства, производительность системы, нейрон-ная сеть, обучение, выборка.
Received 28.12.18
268
Optimizing local feature description andmatching for realtime video sequence object
detectionKatsiaryna Halavataya, Vasili Sadov
Belarussian State UniversityMinsk, Belarus
[email protected], [email protected]
Abstract—The paper proposes an algorithm for localfeature extraction, description and comparison on colorimages for semantic video sequence processing. One ofthe main problems in implementation of such an algo-rithm is its ability to work within realtime constraints.Asymptotic computational complexity for proposed algo-rithm is determined and local performance optimizationsare introduced in order to enhance processing time. Theoptimized algorithm is able to compute local feature vectorsand compare them across video frames in realtime, whichsimplifies further semantic analysis.
Keywords—image processing, image feature extraction,computer vision, algorithm optimization, realtime objectdetection
I. INTRODUCTION
Local image feature extraction, description and matchingalgorithms are the main building blocks of a wide range ofimage processing techniques for computer vision and visualsemantic analysis problems, including image classification,object detection, bundle adjustment, object tracking and visualflow, 3D scene reconstruction, shape and texture extraction, andmany others [1], [2].
The main problem that feature extraction aims to solve isthat image pixel brightness values, per se, are poorly suitedas basis for direct semantic analysis – by itself, a single pixelconveys very little information about the actual contents of theimage. When used as a feature, the pixel value states that thisparticular point of the image has a concrete color value, butthe implication of a specific visual cue cannot be reversed –for instance, a stop sign appearing in the upper left cornerof the image implies that one of the pixels will be red, butthe inverse is not true – one single red pixel in the upperleft corner doesn’t mean that this part of the image contains astop sign. In essence, this example illustrates how raster imagerepresentation is completely different from the actual semanticrepresentation of the objects that can be used as a basis for anautomated decision-making processes [3].
The premise of feature extraction is the fact that actualsemantic analysis based on visual representation is done basednot by individual color values, but rather by meaningful higher-order features and unique characteristics associated with objectsrepresented on the image. For instance, one of the definingsemantic features of stop sign is the octagonal shape. For aproper semantic analysis it is required to create a concretemeasure that correlates with how much similar to an octagona particular group of pixels is. It is obvious that this is veryhard when using raw pixel values, because the particular set of
transforms and combinations required to create an appropriatemeasure can vary greatly from image to image.
This limitation means that raw image data (i.e. pixel bright-ness values or CCD/CMOS image sensor jot charges) is veryhard to analyze in the context of various classification andobject detection problems with classical approaches, like naiveBayesian classifiers or regular feedforward neural networks[3], [4]. For instance, when using a single pixel value as aninput feature for an artificial neural network-based classifier,it is usually hard to measure how much this pixel shouldactually contribute to the output both during training and duringclassification, no matter the weights assigned to them.
Image feature extraction is a set of methods for transformingraw image data (pixel brightness values) into an alternate rep-resentation that is, in turn, better suited as a basis for semanticanalysis in the context of a particular domain problem. Featureextraction as a process can be accomplished by the combinationof feature detector and feature descriptor algorithms [5]–[7].
II. IMAGE FEATURE DETECTORS AND DESECRIPTORS
Image feature detector d(I) (also called feature extrac-tor or keypoint detector) is an algorithm that, given animage, produces a set of coordinates of feature points (orkeypoints) of this image:
d(I) : ∀w, h ∈ N|C(w×h) → (i, j), i ∈ 1, w, j ∈ 1, h, (1)
where I is the input image, w, h – integers representingimage width and height, N – natural numbers set, C –color depth (set of all possible color values), C(w×h) –set of all images. More formally, feature detector can berepresented as a predicate:
∀w, h ∈ N|dp(i, j, I) : (1, w)× (1, h)× C(w×h) → B, (2)
where B = false; true – a Boolean domain. Thatis, a feature detector can be implemented as a functionthat, for any point (i, j) of an image I with color depthC, determines whether this point is, in fact, a keypoint(dp(i, j, I) = true) or not (dp(i, j, I) = false). Basedon this, the set of keypoints in (1) can be determinedusing (2) by applying a predicate over every coordinateset:
d(I) = (i, j) : dp(i, j, I) = true|i ∈ 1, w, j ∈ 1, h. (3)
269
A feature descriptor f is a projection of any point pIijof any image I to an n-dimensional metric vector spaceF :
f(pIij) = ~vIij ∈ F (4)
Since vector space F is also a metric space, anappropriate metric is defined on it:
m : F × F → R (5)
Two arbitary points, pI1 of image I1 and pI2 of imageI2, are considered similar by a feature descriptor (4)if their feature vectors ~v1 = f(pI1) and ~v2 = f(pI2)are similar by measure of metric (5), or m(~v1, ~v2) < t.The threshold t is selected based on a specific descriptorimplementation.
While there is no concrete definition as to what exactlymakes an image point a feature, there is a numberof desireable properties for good feature detectors anddescriptors. Specifically, feature detectors and descriptorsmust be invariant (to a certain degree) to displacement,linear scale-space transforms like rotation, shift, skew,etc., and to some of the common non-linear transformslike perspective shifts and distortion [5].
Detector transform invariance means that any point ofthe image that was classified as a keypoint before thetransform must continue being recognized as a keypointafter it. This is mainly achieved by analyzing a certainsurrounding area of a keypoint in rotationally-invariantmanner (i.e. circular traversal) and on different levels ofsubsampling.
Descriptor transform invariance means that descriptorvectors of the same point before and after transformmust differ, as calculated by metric m, by a smallmargin. This is achieved by including only spatially-invariant information into the descriptor and makingmetric calculation process circularly agnostic.
III. FEATURE DETECTOR AND DESCRIPTOR BASEDON MIDPOINT CIRCLE TRAVERSAL
There are many known approaches to both featuredetection and description. Some of them treat detectionand description as two disjoint operations, while otherscan build a keypoint descriptor as a by-product of deter-mining whether the point should be treated as keypointor not [5]–[7].
One of the main disadvantages of most keypointdetection and description algorithms is their rigidity andinability to adapt in terms of input parameters. Knownmethods usually have a parametric threshold that shouldbe predetermined before feature extraction is carriedout, generating spurious point clouds on one image andhardly detecting any on the others if the threshold isoff. Moreover, these methods tend to be over-relianton sharp brightness shifts, while in reality such shiftsmostly occur as artifacts (like sharp light reflections oroptics chromatic abberations). Finally, most of the known
methods are poorly suited for real-time usages. Whilecertain keypoint descriptors, like FAST, are designed tocarry out keypoint detection in the most efficient manner,they commonly forego the usage of information used toclassify a point to further use it in keypoint description,requiring more complex and computationally expensivekeypoint descriptor algorithms like ORB.
The proposed method is designed to be adaptive andable to perform keypoint detection and description inwith a single pass.
Input of the algorithm is an image I of width w andheight h with 3 color channels (RGB colorspace).
The first step is the selection of image traversal step.A common way to do it is to apply a logarithmic scaleon an input image size:
sg ∈ Nsg ∝ log(w · h)
(6)
This allows to reduce the number of exess points inproximity of a larger feature and makes the algorithmless susceptible to scale transforms.
After that, a one-channel transform is applied to theimage. While it is possible to use common grayscaletransforms like YUV colorspace Y-component formula,doing so usually removes too much of color distributionfrom the image. A proposed solution is to use Prin-cipal Component Analysis (PCA) while treating imagecolor values on 3-component representation (red, greenand blue channels, respectively) as input features in 3-dimensional feature space. Applied to image colorspace,PCA will generate a new basis with 3 pseudocolor direc-tions. First direction corresponds to the largest varianceof the image, which is, for most of the real-world images,a shift from darker to brighter colors – that is, firstcomponent of the new colorspace in PCA is usuallyvery close to Y direction of YUV colorspace. Based onactual variance numbers, however, a second componentmight be preferable, the one corresponding to the second-largest variance distribution of colorspace. This is usuallya shift from one predominant color of the image tothe other. In some specific areas, like medical imageprocessing, color hue may provide more destinguishablefeature space compared to plain grayscale brightness. Asa general rule, if the variance of the second componentis above 20% of the total image variance, a second PCAcomponent should be considered for feature extraction.It is important to note, however, that, when comparingfeature descriptors across images, descriptors based offfirst and second components tend to differ significantly,so the same component should be used on both images.
After the step is chosen and image is transformedto grayscale, a set of traversal radiuses ri must bechosen. A common choise is 3, 5, 7 pixels, with 9pixels added for hevily distorted images. The maximumradius rmax = max
iri defines a padding for image
270
traversai, that is an image is traversed from rmax tow − rmax horizontally and from rmax to h − rmax
vertically with the step sg .For every point (i, j) traversed, a Bresenham (mid-
point) algorithm is used to construct a circle around thepoint at every specified radius ri. The result for radiusrk is an ordered set of point coordinates prkn . Beforethe circle traversal, a starting point is determined as apoint where brightness values change compared to thenext pixel is the greatest across the entire circle. This isdone to mitigate the influance starting point has whencomparing features before and after rotation transforms.
The goal of Bresenham circle traversal is to determinesegment configuration. The first pixel is assumed tobelong to the first segment: p(rk)1 ∈ S0. After that, everynext pixel brightness of the circle in order is comparedto the brightness of the first pixel of a current segment.When the brightness exceeds a certain threshold tseg ,the pixel is considered a beginning of a new segment.The process iteratively continues until the point circle isconfined:
∆P (j)n = |I(prkn )− I(S
(j−1)0 )| (7)
Pn ∈ S(j) if ∆P
(j)n ≥ tseg
Pn ∈ S(j−1) if ∆P(j)n < tseg
(8)
Brightness difference (7) across two neighbour pointsin Bresenham circle is used in (8) to determine thebeginning of a new segment. Artificial segment edgeintroduced by the first pixel is mostly mitigated bycorrect selection of the first pixel.
The segment configuration, i.e. the number of seg-ments and their length, constitute a feature of a certainpoint. Variable-length natural-numbered segment spanvector can be used as a feature vector of a certain point.It is also possible to short-circut descriptor evalutation ifa point should not be considered keypoint, in case thenumber of segments or their respective differential shiftis too small on any of the radiuses.
To compare the features, segment configuration simi-larity measure must be created. This process is compli-cated by the fact that segment span vectors have variablelength, so traditional similarity measures like Euclideandistance cannot be applied to them. One way to solve thisproblem is to introduce a simple binary representationof a feature vector. For this descriptor, each segment ofa specific length constitutes a span of two-bit numbersof the same length, and each segment edge changesthe segment two-bit representation with the followingcircular pattern: 00 → 01 → 11 → 10 → 00. Forexample, for an 12-point circle (radius 3) with segmentsof length 3, 7, 3, the binary representation will be 24-bitstring – 00 00 00 01 01 01 01 01 01 01 11 11 11. Thespecific pattern is chosen in such a way that Hammingdistance (the number of differing bits) increases by 1 foreach two-bit marker. To illustrate this, let’s consider a
12-point circle with segments of length 3, 6, 1, 3. Its24-bit descriptor will be the following: 00 00 00 01 0101 01 01 01 11 10 10 10. The difference beetween theprevious 3, 7, 3 segment is the introduction of anothersmall segment, which means that configurations remainrelatively similar. If a 1-bit representation was used, anextra segment would invert the rest of the string, resultingin a completely different 12-bit descriptor with largeHamming distance. With 2 bits per segment span, only"half" is inverted. In our example, Hamming distancebetween feature vectors is 4.
The set of keypoints and their respective descriptorsfor each radius can be used for further processing intemplate-based object detection algorithms and 3D re-construction tasks.
IV. REALTIME OPTIMIZATION
Realtime video sequence processing puts a time con-straint on feature detector and descriptor performance.
Generally accepted framerate for video sequence im-age rendering is 24fps (frames per second). That meansthat it should take around 40ms for an entire frame torender. If the information can be presented as a HUD(heads-up display) overlay over existing video sequencewith visual cues (as is the case with object detectionand localization, for instance), it is generally acceptableto have HUD re-render at lower framerates, with timesof up to 100ms per frame. Still, this constraint is quitesevere in terms of computational complexity of algo-rithms involved in common computer vision tasks. Forinstance, a well-calibrated feature-based binary classifierneeds as much as 5ms to process a single sliding windowframe, and needs to run the classification up to 10-15sliding window positions per frame to correctly localizean object. In practice, this means that, independently offurther processing and semantic analysis, feature detectorand descriptor should produce extracted feature vectorswithin 20ms for acceptable performance that can still bepercieved as realtime [4], [8], [9].
It’s easy to illustrate how the feature extraction pro-cedure (7) - (8) has a linear asymptotic computationalcomplexity. PCA transform requires mean and standarddeviation calculation, which require a full image pass andhave the complexity of O(n) each. The transform itselfis a O(p3) complexity problem, where p is the numberof input features: for color images, p = 3 = const,i.e. it doesn’t depend on the input image size. Detectionrequires a full traversal of an entire image, which is alinear O(n) complexity compound problem. For eachpixel, a constant number of operations must be performedto construct a descriptor, so the complexity of singlepoint analysis is O(nr) where nr = const is thenumber of radiuses used for descriptor generation. Theresulting complexity is, in terms of big-O notation, isO(2n+ p3 + nrn) = O(n).
271
The nature of feature extraction process means that it’snot possible to further reduce asymptotic complexity, sothe optimizations must be performed during key stagesof the algorithm.
PCA computation is generally quite costly, since itrequires the calculation of eigenvectors and eigenvaluesof color feature matrix. However, when working witha video sequence, it can be safely assumed that PCAtransform basis will be quite similar as long as the scenedoesn’t change drastically. In practice, it means thatthe same PCA transform can be used across the seriesof similar frames, for times up to several seconds ofvideo sequence. So the first optimization is to skip PCAcomputation and instead use previous transform matricesacross several frames. It is also possible to performthis computation in parallel to the main data streamon a single frame in isolation, and replace the currenttransform once PCA is calculated, further reducing frameprocessing time.
Detection itself requires a full image traversal at theminimum [10]. The traversal step grid introduced in (6)means that a portion of image’s pixels is skipped already.To further increase processing performance, it’s possibleto statistically analyze the keypoint detection distribu-tion. With adaptive threshold value the total number ofkeypoints on the image takes up to 10% of all the imagepixels, which means that bail-out optimizations mightactually increase processing speed. Bail-out optimiza-tions are a way to short-circut the evaluation in caseof a negative answer by performing a short check. Incase of Bresenham circle descriptor, one common wayto discern a keypoint is to select only those points wherenumber of segments is large enough. It is possible totraverse the smallest radius value only, and mark thepixel as non-keypoint early if it doesn’t have enoughsegments, thus avoiding additional traversals on largerraduises. Since this happens for most of the image pixels(around 90%), this optimization provides a noticeableperformance boost.
V. RESULTS AND CONCLUSION
The proposed algorithm allows for efficient feature extrac-tion from color images. It is able to both determine a set ofkeypoints on the image and calculate their respective descriptorvalues in one pass, while also providing a bit string descriptorsvalues that can be compared very fast using Hamming distanceacross several frames.
The performance optimizations have been evaluated for aprocessing task of feature extraction in realtime over 1 minuteof FullHD video sequence. The results are presented in tableI.
As can be seen from the results, the goal of realtime perfor-mance optimization of midpoint feature extraction algorithmwas successfully met – the framerate constraint of 20ms perframe for HUD processing tasks was fulfilled using Midpointcircle traversal algorithm with bail-out optimization and stag-gered PCA computation across 24 frames (basis recalculatedevery second). For comparison, FAST feature detector and ORB
Table IMEAN FRAME PROCESSING TIME t FOR FEATURE EXTRACTION ONFULLHD (1920 × 1080) 24FPS VIDEO SEQUENCE OVER 1 MINUTE
(1440 FRAMES TOTAL)
Algorithm t, msFAST & ORB 27.35
Midpoint without optimizations 35.02Midpoint with staggered PCA 28.94
Midpoint with bail-out 23.14Midpoint with bail-out & staggered PCA 19.41
feature descriptor performance was also considered. While bothof those algorithms have linear asymptotic complexity, theinformation used in FAST for keypoint detection is not used fordescriptor generation, as ORB is a separate algorithm; thus, thealgorithms require twice the amount of image traversals, whichimpacts performance on larger images, like FullHD-scale imageused to evaluate the performance.
REFERENCES
[1] R. S. Choras Image Feature Extraction Techniques and Their Applicationsfor CBIR and Biometrics Systems. International Journal of Biology andBiomedical Engineering, 2007, vol. 1, no. 1, pp. 6-16.
[2] L. G. Shapiro, G .C. Stockman Computer Vision, New Jersey, Prentice-Hall,2001. 608 p.
[3] Z. Shi, S. Vadera, A. Aamodt Image Semantic Analysis and Understanding.IFIP Advances in Information and Communication Technology, 2010, vol.340, pp. 4-5.
[4] L. Qin, L. Kang Application of Video Scene Semantic RecognitionTechnology in Smart Video. Technical Gazette, 2018, vol. 25, no. 5, pp.1429-1436.
[5] D. Lowe, Distinctive image features from scale-invariant keypoints. Inter-national Journal of Computer Vision, 2004, vol. 60, no. 2, pp. 91–110.
[6] K. M. Yi, E. Trulls, V. Lepetit, P. Fua Lift: Learned invariant featuretransform. European Conference on Computer Vision, 2016, pp. 467-483.
[7] E. Rublee, V. Rabaud, K. Konolige, G. Bradski ORB: an efficient alterna-tive to SIFT or SURF. IEEE International Conference on Computer Vision(ICCV), 2011, pp. 2564-2571.
[8] S. Baier, Y. Ma, V. Tresp Improving Information Extraction from Imageswith Learned Semantic Models. 27-th International Joint Conference onArtificial Intelligence (IJCAI-18), 2018, pp. 5214-5218.
[9] I. Simon, N. Snavely, S. M. Seitz Scene summarization for online imagecollections. IEEE International Conference on Computer Vision (ICCV),2007, pp. 1-8.
[10] R. Girshick, J. Donahue, T. Darrell, J. Malik Rich feature hierarchies foraccurate object detection and semantic segmentation. IEEE conference oncomputer vision and pattern recognition (CVPR), 2014, pp. 580-587.
ОПТИМИЗАЦИЯ АЛГОРИТМОВ ОПИСАНИЯ ИСРАВНЕНИЯ ЛОКАЛЬНЫХ ПРИЗНАКОВИЗОБРАЖЕНИЙ ПРИ ДЕТЕКТИРОВАНИИ
ОБЪЕКТОВ НА ВИДЕОПОСЛЕДОВАТЕЛЬНОСТЯХВ РЕАЛЬНОМ ВРЕМЕНИ
Головатая Е.А., Садов В.С.
В работе предложен алгоритм извлечения, описания исравнения локальных признаков цветных изображений длясемантической обработки видеопоследовательностей. Однаиз решаемых задач - поддержка работы алгоритма в реальномвремени. Для оценки и улучшения производительности про-ведён анализ асимптотической вычислительной сложности,а также предложены оптимизации. Оптимизированный ал-горитм позволяет вычислять вектора локальных признакови сравнивать их между кадрами, а также может служитьосновой для дальнейшего семантического анализа.
Received 09.01.19
272
Effective Algorithm for Object Detection in theVideo Stream for ARM Architectures
Kanstantsin Kurachka, Ihar NestsiareniaSukhoi State Technical University of Gomel
Gomel, [email protected], [email protected]
Abstract—An algorithm is presented that allows detectingmoving objects in a video stream at a fixed cameraposition. The algorithm is characterized by low resourceconsumption, what makes it possible to use it in ARM ar-chitectures [1] or for data pre-processing on client devices.The algorithm performs the following steps: image scaling,clipping background, and detection of objects. Prototypealgorithm implemented in Python [2] using the OpenCVlibrary. It was tested on a single-board computer RaspberryPI 3 [3], showed a performance of 20 FPS with an inputstream frame size of 1920x1280 pixels.
Keywords—detection, video processing, classification
I. INTRODUCTION
Currently, neural networks are widely used for digital imageprocessing, but they usually require significant computationalresources. Therefore, in practice, it is advisable to combinethe using of neural networks with algorithms that do not usetraining, which will in some cases increase the speed of solvingproblems and reduce resource consumption [6].
Neural networks show a high degree of classification of alarge number of different objects, but the performance is 6-8 FPS [5], [7]. The solution to this problem is to use high-performance hardware with the GPU, it is not always possibleand appropriate on grounds of the cost and energy efficiency.One of the modern practical tasks is the real time analysis ofstreaming video data. For such problems, the accuracy of thealgorithm can be neglected. There is no need to ensure thedetection of objects at each frame, it is necessary to achievethat at least one of a series of frames of the object is detected.
This kind of task includes detection of cars using surveil-lance cameras. There are two approaches to solving this prob-lem, the first one is to process completely on the portabledevice, the second one is to send all video stream to the serverfor centralized analysis and recognition. The first approachdepends on resources, it is not always possible to use high-performance devices on the end users of the system. In thesecond approach, it is necessary to provide a reliable and securecommunication channel with high performance for connectingclient devices to the data processing server, which in modernconditions is also costly and sometimes completely unrealizablefor a number of subjective reasons. Providing a similar channelfor multiple connections is very expensive, and the cost willincrease with connecting new devices, which indicates theproblems of scaling such a solution.
A possible solution could be the development of algorithmsthat reduce requirements of throughput and the performanceof the computing server by performing preliminary partialprocessing of the video stream on the end devices. This paperproposes using a combination of transformations and filtersto significantly reduce the amount of data transmitted to the
processing server. On the resulting images, you can quicklysearch for objects using cascades [8].
On the client side, cheap single-board computers with ARMarchitecture [3] are offered as hardware for pre-processing. lowpower, e.g., Raspberry PI 3 [2].
II. GENERAL ALGORITHM
The camera receives the data that needs to be processed. Toreduce video traffic on the client-server, the input video streamis sent to a single-board computer, where it is pre-processedaccording to a multi-level scheme. Levels are represented bythe black box model and are interconnected only by input andoutput data. This approach allows the use of conveyor parallelprocessing, thereby ensuring a uniform load of the existingcores and the highest performance [4].
The diagram of processing is presented in Fig. 1 and includesthe following steps:
Figure 1. General algorithm diagram
• Scaling – provides camping downsampling as to detectthe object is not necessary for high definition.
• Background subtraction – allows you to simplify the taskof searching for objects since all objects that do not movefor a long time will be removed from the image.
• Object detection in the image – the direct application ofsearch algorithms and the formation of the result in theform of coordinates of rectangular regions.
• Coordinate projection – the coordinates of the regions areconverted according to the original image size.
At the first stage, the image size is reduced and it isconverted to shades of gray:
[M ]norm = G([M ]) (1)
where M – is the original image, G – is the normalizationoperator of the input image.
III. BACKGROUND SUBTRACTION
To improve the quality of image detection, it is possibleto cut off static unchangeable parts (background), we takeadvantage of the fact that cameras are placed statically. This
273
gives a significant advantage when searching for objects andtheir recognition. The previous frames can be used to selectthe background, the part which is not changing some period oftime. This part of the video frame can be removed, what willreduce the amount of data to analyze. Thus, the backgroundsubtraction is performed based on the available set of images(2).
[M ]FG =‖ [M ]norm − ∩i[M ]i ‖ (2)
It seems that for detecting moving objects of a given type,there are enough background subtractions algorithms [9], butthis is not so, because it is necessary to notice not any movingobjects, but only a certain type. For example, for the task ofdetecting moving cars, the classifier should not react to otherroad users (pedestrians, cyclists, etc.), and in general to foreignobjects that are in the frame.
Before using the subtraction of the background operation,an additional configuration of the scene is required to selectand classify sections of the road: curb, pavement, lanes in bothdirections, etc.
In this case, at the preliminary stage, the boundaries of theanalyzed areas of the video stream will be determined. In thiscase, it is possible to perform parallel processing of each ofthe areas found using cascades. The use of decomposition ofthe analyzed scene will reduce the amount of data processed,increase the quality and speed of the search.
Shadows and noise in the image can complicate the work ofthe background subtraction algorithm. For example, a shadowmoving together with objects may significantly reduce thequality of detection or even lead to the impossibility of recog-nition and classification of an object. Therefore, it is necessaryto perform image transformation in such a way that all the"surroundings" of the desired object are removed, even if thisoperation will result in distortion of the object itself.
Thus, in formula (2), it is required to specify not onlythe metric used but also the set [M ]i. All this is determineddepending on the selected background subtraction algorithm.When choosing an algorithm, the main criteria, determinedby the need to perform on low-performance devices, were itsresource intensity, speed, the accuracy of correctly detectedimages after background clipping.
The following algorithms were analyzed:• MOG – based on a Gaussian mixture model [10].• MOG2 – based on MOG ideas, the main feature is that the
algorithm chooses an approximate number of Gaussiandistributions for each pixel [11], [12].
• GMG – algorithm combines a statistical evaluation ofthe original image and pixel-by-pixel Bayesian segmenta-tion [13]. After evaluating the first frames, the algorithmfinds the objects separating them from the background.
• KNN – based on the k-means algorithm [14].As a result of computational experiments, at the preliminary
stage, methods based on the k-means algorithm were excluded,since they were less effective according to the specified criteria.
MOG uses a number of Gaussian distributions, also calledGaussian components, to simulate the background of pixels.Each pixel has its own set of Gaussian components. Eachcomponent needs three parameters: the average (backgroundintensity), weight (as an indicator of the significance of thecomponent) and the standard deviation of intensity [15], [16].
MOG2 is an improved version of the MOG algorithm, and,due to the possibility of adaptively adjusting the number ofdistributions for each pixel, is more resistant to changes inlighting on the scene. This is a significant advantage, making itpossible to use this algorithm on external observation cameras.For the task of detecting cars is an important condition. The
algorithm should work steadily throughout the day and whenthe weather conditions change.
As a result, formula (2) was converted to the form:
[M ]BGt (xi) = p(xi|BG) =
K∑
k=1
πikN(µi
k, σI) (3)
where K - the number of Gaussian components, each with aweight ωi,t and intensity µi,t and standard deviation σi,t, N –normal distribution.
The following parameters for the MOG2 algorithm werecalculated empirically: the number of Gaussians is 2, thefiltering of shadows, and the complexity reduction thresholdthat determines the number of examples needed to determinewhether an image component exists or not. Such values allowyou to deal with various noises in the video, shadows, cameravibration. At the same time, which is very important, thepossibility of classifying objects does not deteriorate. Foranalysis, 10 frames were used with the quality of shooting 20FPS. This number of frames is enough to detect the movementof objects and allows you to select the background (Fig. 2).
Figure 2. Background subtraction
IV. OBJECT DETECTION USING CASCADES
Algorithms that use the idea of cascades show their effec-tiveness when searching for one type of objects in an image[17].
The basis of these algorithms are the Haar primitives [8],which represent the discretization of a given rectangular regioninto sets of heterogeneous rectangular subregions. In the origi-nal version of the algorithm, only primitives without turns wereused, and to calculate the feature, the difference of the sum ofthe brightness of one region from the sum of the brightness ofthe other subregion. When developing the approach, we beganto use oblique primitives, instead of calculating the differencein brightness, it was proposed to assign a separate weight foreach subdomain and calculate weighted sums of features fordifferent types of areas. Thus, a set of classifiers is compiled,each of which is assigned a weight during training; training isusually performed using the AdaBoost algorithm.
The advantage of the Viola-Jones algorithm is the ability toquickly calculate signs using the integral representation of theimage. This means that to calculate the sum of the intensitiesof pixels in the selected area, you can use 4 values of thesum of intensities s1, s2, s3, s4 from which you can calculatethe value in a given area 4. The construction of the integralrepresentation of the image has complexity O(n) where n isthe number of pixels in the image. The amount of light in thegiven area 5.
D = s1 + s4 − (s2 + s3), (4)
si =
∫∫
S
I(x, y)dxdy ≈xSi∑
x=0
ySi∑
y=0
I(x, y), (5)
where S is the sampling primitive, I(x, y) – pixel intensity withcoordinate (x, y) xSi – boundary coordinate x of rectangle SiySi – boundary coordinate y of rectangle Si i = 1, 4.
274
To determine class membership, a classification functionis introduced in each stage. Each "weak" classifier producestwo values, depending on whether the value of the attributebelonging to this classifier is greater or less than the specifiedthreshold. At the end, the sum of the values of "weak" classi-fiers is compared with the cascade threshold and the decisionis made whether the object is found or not by this cascade 6. Inthe process of learning a lot of classifiers are built and weightsare selected for each of them. The resulting classifier is used todetermine whether the site belongs to the desired object class 7.
Ψ(x) = sign[
k∑
i=1
ωiRi(x)] (6)
where is "weak" classifier, ωi – weight of i-th classifier; k –the number of classifiers.
As a result of background subtraction, a filtered imageis obtained, which is divided into regions r using the E()operator:
r = E([M ]FG),
each found region, we introduce a classifier using 4, whichreturns the probability of finding an object in the region.
Ψ(oi) = 1, ifoi ∈ ri (7)
where: oi – the desired object.The use of only signs of Haar makes it possible to find
more than 80% of the objects. Change video angle, scale, etc.the quality of recognition with the use of cascades sharplydeteriorates [18]. In practical terms, the number of signsincreases nonlinearly, which significantly reduces the speedof the algorithm. In this case, you need to either limit thepossible situations, for example, only for a certain cameraposition (distance, and rotation angle), or add additional videoprocessing to reduce the set of features without presentingspecial video requirements. The proposed modification of thealgorithm is undemanding to the initial configuration.
This algorithm is undemanding to computing resources andcan be implemented on almost any modern ARM processor,which allows the proposed concept to be used to build complexrecognition systems.
V. DATA PREPARATION AND TRAIN CLASSIFIER
High-quality data preparation is the most important step inthe development of the classifier. The algorithm will workeffectively only when there is a lot of data collected thatmaximally covers all possible variations of the objects beingclassified. In relation to the task of detecting cars, you needto select photos taken from different angles and from differentangles. Also in the training set should get cars with differentcolor shades.
To train the cascade on "raw" images, you need to preparemore than a thousand samples, with an approximately uniformdistribution of different combinations of colors, models andangles.
Due to the non-linear dependence of computational com-plexity and quality of classification on the number of features,combining all the properties of objects in one classifier leads toan increase in resource intensity and a drop in the recognitionrate of the desired object. Therefore, by applying backgroundsubtraction as a side effect, very simplified car images areobtained, which makes it possible to neglect the different colorshades of the original images.
Thus, it is proposed to introduce an additional step inpreparing the training data: the source video stream is pro-cessed by the background clipping algorithm. And to achieve
maximum accuracy of the classifier, you need to use the sameconfigurations for background clipping, which will be used ina working system. An example of an image before and afterapplying a filter is shown in Fig. 3.
To train a classifier, it is necessary to assemble a set ofnegative examples - background images, and objects that shouldnot be detected.
Figure 3. Examples of images for training, on the left are positiveexamples, on the right are negative
For classifier operation, it is recommended to use at least 500positive examples of images, 2-4 times more negative ones.It is also recommended to use augmentation, with reflectionhorizontally, scaling, and turning in the limit of 15 degrees.
VI. EVALUATION
The article presents an algorithm that is not demanding ofcomputational resources, characterized in that it is a combi-nation of image processing and detection algorithms, whichallows video to be processed in real time. The first is theclipping of the background based on the previous frames, then -learning the cascade on the resulting images. After clipping thebackground image, a lot of noise is produced, but the vehicleshave expressive boundaries, resulting in a simple set of features.
The combination of algorithms allows to achieve high pre-cision detection of passenger cars (about 95%) and process upto 20 frames per second with a resolution of 1920x1280. Anexample of detection is shown in Fig. 4.
Figure 4. Detection Results
The proposed algorithm can be used to build traffic monitor-ing systems, as well as subsequent statistics collection, whichwill provide the most relevant information about the state of acity’s roads at any time . The introduction of monitoring sys-tems makes it possible to apply various mathematical models tooptimize traffic, because automatic systems for collecting statis-tics from highways will allow you to accumulate a sufficientamount of information that can be used in conjunction withmachine learning algorithms in order to improve throughputand safety of highways. Also, the collected information will
275
be very valuable when making changes in traffic (for example,closing the street, or expanding it), the information collectedwill always be relevant, which means modeling changes intraffic will provide the greatest efficiency.
The resulting algorithm can be used to build statisticscollection systems with preliminary data processing on thedevice. Low algorithm requirements make it possible to uselow-power devices.
REFERENCES
[1] Seal David, ARM architecture reference manual, Pearson Educa-tion, 2001
[2] Python. Available at: https://www.python.org/ (accessed 2018,Dec)
[3] Raspberry Pi 3 model b. Available at:https://www.raspberrypi.org/products/raspberry-pi-3-model-b/(accessed 2018, Dec)
[4] Kurochka K. S., Safonau I. V. Medical Images Indexing andRetrieval in a Distributed Computing Environment //Journal ofAutomation and Information Sciences, 2010, vol. 42, No 5
[5] Kurachka K. S., Tsalka I. Vertebrae detection in X-ray imagesbased on deep convolutional neural networks //Informatics, 2017IEEE 14th International Scientific Conference on IEEE, 2017, pp.194-196.
[6] Xun Wang, Jie Sun, and Haoyu Peng. Foreground object detectingalgorithm based on mixtureof gaussian and kalman filter in videosurveillance.JCP, 8(3):693–700, 2013.
[7] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Fasterr-cnn: Towards real-timeobject detection with region proposalnetworks. In Advances in neural information processingsystems,pages 91–99, 2015.
[8] Paul Viola and Michael Jones. Rapid object detection usinga boosted cascade of simple features.In Computer Vision andPattern Recognition, 2001. CVPR 2001. Proceedings of the 2001IEEEComputer Society Conference on, volume 1, pages I–I.IEEE, 2001.
[9] Temirlan Z. Kamilla A. Vehicle detecting, tracking and countingfor traffic control systems.icp2015, 2015.
[10] Pakorn KaewTraKulPong and Richard Bowden. An im-proved adaptive background mixturemodel for real-time track-ing with shadow detection. Video-based surveillance systems,1:135–144,2002.
[11] Zoran Zivkovic. Improved adaptive gaussian mixture model forbackground subtraction. InPattern Recognition, 2004. ICPR 2004.Proceedings of the 17th International Conference on,volume 2,pages 28–31. IEEE, 2004.
[12] Zoran Zivkovic and Ferdinand Van Der Heijden. Efficient adap-tive density estimation per imagepixel for the task of backgroundsubtraction.Pattern recognition letters, 27(7):773–780, 2006.
[13] Andrew B Godbehere, Akihiro Matsukawa, and Ken Goldberg.Visual tracking of humanvisitors under variable-lighting condi-tions for a responsive audio art installation. In AmericanControlConference (ACC), 2012, pages 4305–4312. IEEE, 2012.
[14] Darren Butler, Sridha Sridharan, and VM Jr Bove. Real-timeadaptive background segmentation.In Acoustics, Speech, and Sig-nal Processing, 2003. Proceedings.(ICASSP’03). 2003 IEEEInter-national Conference on, volume 3, pages III–349. IEEE, 2003.
[15] Hamed Tabkhi, Robert Bushey, and Gunar Schirner. Algorithmand architecture co-design ofmixture of gaussian (mog) back-ground subtraction for embedded vision. In Signals, SystemsandComputers, 2013 Asilomar Conference on, pages 1815–1820.IEEE, 2013.
[16] Nils Stahl, Niklas Bergstr om, and Masatoshi Ishikawa. Exploit-ing high-speed sequences forbackground subtraction. In PatternRecognition (ACPR), 2015 3rd IAPR Asian Conference on,pages106–110. IEEE, 2015.
[17] Paul Viola and Michael J Jones. Robust real-time face detection.International journal ofcomputer vision, 57(2):137–154, 2004.
[18] M Oliveira and V Santos. Automatic detection of cars in realroads using haar-like features.Department of Mechanical Engi-neering, University ofAveiro, 3810, 2008
ЭФФЕКТИВНЫЙ АЛГОРИТМДЕТЕКТИРОВАНИЯ ОБЪЕКТОВ НА
ВИДЕОПОТОКЕ АДАПТИРОВАННЫЙ ДЛЯARM АРХИТЕКТУРЫ
Курочка К.С., Нестереня И.Г.
Представлен алгоритм, позволяющий детектироватьдвижущиеся объекты в видеопотоке при фиксиро-ванном положении камеры и отличающийся низкойресурсоёмкостью, что позволяет его использовать вARM архитектурах [1] или для предварительной обра-ботки данных на конечных (клиентских) устройствах.Алгоритм реализует следующие этапы: масштабирова-ния изображения, отсечения фона, и детектированияобъектов. Тестирование алгоритма, реализованногона языке Python [2] и с использованием библиотекиOpenCV, на одноплатном компьютере Raspberry PI3 [3] показало производительность – 20 FPS приразмере кадра входного потока 1920x1280 точек.
Received 29.12.18
276
Development of neural network-basedconsultant recognition method for determining
posture and behaviorRozaliev V.L., Alekseev A.V., Ulyev A.D., Orlova Y.A.
Volgograd State Technical UniversityVolgograd, Russia
[email protected], [email protected],[email protected], [email protected]
Petrovsky A.B., Zaboleeva-Zotova A.V.Federal Research Center
Computer Sciences and Control, RASMoscow, Russia
[email protected], [email protected]
Abstract—The article presents the method recognitionof sales consultants on the basis of a neural networkto determine the posture and clarifying algorithms, alsomethods of monitoring the behavior of the seller-consultantand analysis of its interaction with the buyer. A briefreview of the analog systems is given. The description ofthe proposed method is presented, the obtained results andways of improvement are shown.
Keywords—neural network, artificial intelligence, recog-nition of human pose, behavior monitoring.
I. INTRODUCTION
The modern era is characterized by a transition fromthe economy of producers to the economy of consumers.In the conditions of toughening competition in the sphereof trade and rendering services, client-oriented servicesacquire special importance.
The main problem of introducing such services is thehuman factor, control of which is problematic due to thelack of ready-made software products.
Ensuring the proper quality of service delivery be-comes the main objective of the market strategy forbusiness development.
To improve the quality of service, it is proposed todevelop and implement a software product to monitorthe activities of consultant salesmen through the analysisof their work with the use of equipment for video fixing[10].
The work of the software product is based on the algo-rithm Pose Estimator [3], which allows you to determinethe position of a person, clarifying algorithms, auxiliaryneural networks that help to identify the seller-consultant,and also determine the quality of services provided tothem [2].
II. THE PROPOSED METHODOLOGY
To solve the problem, we propose to use a cascade oftwo neural networks:
• Fast convolutional neural network FastPoseEstima-tor, trained on mobilenet architecture;
• Algorithm of stabilization of "key" points, allowingdetermine those points of the body, which could notrecognize the neural network;
• Neural network for determine the behavior of thesales assistant and store employee.
III. THE FIRST STAGE, THE USE OF THE NEURALNETWORK "FAST POSE ESTIMATOR".
The main task of the neural network is to establisha person’s pose through a nonparametric representationcalled the Part Affinity Fields (PAFs) by developers, tofurther determine the location of the seller’s uniform ofthe consultant (branded T-shirt, cap, etc.).
The main advantage of the neural network is the highspeed of the work, 1 frame in 1-2 seconds on GPU and5-8 seconds on CPU. The main drawback is the decreasein the quality of work when compared with the classicversion of Pose Estimtor.
Input data for the algorithm "PoseEstimator" is agraphic image of the sales consultant, on the output -an image with the selected parts of the human body.
The result of this network can be seen in figure 1.
Figure 1. The result of the Fast Pose Estimator neural network.
277
IV. THE SECOND PHASE, A STABILIZATIONALGORITHM "KEY" POINTS OF THE A PERSON BODY.
The skeleton is built from the starting point locatedunder the throat, then the eyes, shoulders and pelvis arerecognized. From the shoulders completed the hands,from the points of the pelvis of the foot, from the eyesears. Thus, the key point for the construction is the pointunder the neck, then we call it the initial one.
It is necessary to determine the dominant color ofthe t-shirt seller. To improve the quality of work, thefollowing algorithm is proposed for finding the missing"key" points, based on information about the structureof the human structure. Consider one of the options forfinding the missing "key" point of the body.
If only one shoulder of a person is recognized, thesecond shoulder can be completed by depositing an equalsegment from the projection of the starting point on thenormal of the point of the found shoulder to the side ofthe not found shoulder equal to the distance from theprojection of the point to the normal of the point of thefound shoulder to the point of the found shoulder.
Thus, using the information about the structure of thehuman structure it is possible to determine the approx-imate location of the missing points of the shoulders,pelvis points. If there is only a starting point, it isproposed to determine some area of a fixed size on thehuman’s body by depositing the area below the key pointby N pixels. An example of the algorithm is shown infigure 2.
Figure 2. Example of detection of the second shoulder.
V. THE THIRD STAGE, THE DETERMINATION OF THEDOMINANT COLOUR IN THE UNIFORM SECTION.
The main task of this stage is to establish a dominantcolour in the area of the uniform of a person to determineit in the group of sales consultants. Within the frameworkof this algorithm, an image from the "Pose Estimator"with the tops of the human body parts is input. Onthe basis of which there is a selection of the necessaryclothing of a person [4] [5] [6]. To implement thedefinition of dominant colour in an established area, thereare several methods: determining the ratio of a pixel to
a given set of colours and clustering by the k-meansmethod.
In the first method, the image is converted to HSVcolour space, after which all pixels of the image areanalyzed and based on the Hue, Saturation, Value data,the colour is set.
The idea of the k-means method [7] is to minimizethe total quadratic deviation of the cluster points fromthe centers. At the first stage, you select points (three-dimensional RGB space) and determine whether eachpoint belongs to this or that center. Then at each stage,the centers are redefined until a single center is found.
VI. TRAINING THE APPEARANCE OF SALESCONSULTANTS.
To determine only the colour of a person’s clothing isnot enough to classify him as a sales consultant group.It is necessary to take into account the conditions of thedifference in the illumination of the room at differenttimes of the day, as well as the likelihood that theremay be clarified and dark areas in the room. Thus, therecognized colour of the shape can vary.
To solve this problem, it is necessary to teach thesystem all possible colors that can be "read" from theclothing of the seller-consultant.
The operator of the software product at the beginningof work with the program should start the trainingmode, with by means of which prevailing color uniformsrandomly moving around the store sales assistant.
On the basis of the prevailing colors of the clothes isformed an "average dominant color", which is the centralpoint in the formation of the color range for referring aperson to a sales consultant [1].
VII. ANALYSIS OF THE QUALITY OFCOMMUNICATION WITH THE SELLER BY BUYERS.
After the classification of people in the frame to groupsof buyers and sellers, the program automatically controlsthe quality of services rendered by sellers-consultants. Toassess the quality of the seller’s communication with theclient, it is proposed to use a set of algorithms and acascade of neural networks.
The main task of the algorithms is to determine thelocation of the seller next to the buyer, as well as controlover the personalization of the employee’s appeal to thebuyer.
In order to exercise control over the personalizationof the employee’s appeal to the buyer, it is proposed todetermine the “field of view” of people in the frame.The seller must always interact with the buyer, be in the“review” area of the buyer and talk about the benefits ofthe goods. Thus, to accomplish the task, it is necessaryto build a “field of view” based on the location of theeyes and ears obtained from Pose Estimator, after whichthe sector of intersection of these areas and the “angle
278
of interaction” between the buyer and the seller shouldbe determined.
If the areas do not intersect or the “interaction angle”is less than the angle set by the operator of the softwareproduct, it is considered that the seller is near the buyer,but does not interact with it. Examples of definitionalgorithms for the “field of view” are presented in figure3.
To determine the interest and emotionality of the storeemployee when interacting with the buyer, it is proposedto use neural networks that define the above parametersby the store employee behavior: hand speeds, movingaround the store to accompany the buyer, movements toshow goods, etc.
In addition, it is planned to use a separate neuralnetwork, which, by facial expression and behavior, willdetermine customer satisfaction with the services pro-vided by the seller’s. The conclusion about the workseller’s of the store is made on the basis of all the factorsthat determine the interaction with the buyer, after whichat the end of the month an estimate is calculated for eachstore employee.
Figure 3. An example of the definition of "field of view".
VIII. OVERVIEW OF ANALOGUES.
It should be noted that the finished software productsthat allow to solve the problem discussed in this articleare not present. Similar software products perform onlypart of tasks.
The simplest example of intelligent video surveillanceis motion detection. One detector can replace severalvideo surveillance operators. And in the 2000s, the firstvideo analytics systems began to appear, capable ofrecognizing objects and events in the frame. Most ofthe solutions work with face recognition technologies.Solutions in this area include Apple, Facebook, Google,Intel, Microsoft and other technology giants. Surveil-lance systems with automatic passenger identificationare installed in 22 US airports. In Australia, they aredeveloping a biometric system of face recognition andfingerprinting within a program designed to automatepassport and customs control. An interesting project of
NTechLab company showed a system capable of real-time recognition of sex, age and emotions using the im-age from a video camera. The system is able to evaluatethe audience’s reaction in real time, so you can identifythe emotions that visitors experience during presentationsor broadcasts of advertising messages. All NTechLabprojects are built on self-learning neural networks. In oursystem, we do not yet use data on a person’s face. Weplan to process this information at the next stages of theproject development.
In other systems, the object tracking function is used -tracking. The operation of the tracking modules is relatedto the operation of the motion detector. To constructthe trajectories of the movement, a sequential analysisof each frame is carried out, on which moving objectsare present. The simplest implementation of trackingconsiders two frames and builds trajectories along them.First, the movements on the current and previous frameare marked, then, by analyzing the speed, the directionof movement of objects, and also their sizes, the prob-abilities of the transition of objects from one point ofthe trajectory of the previous frame to another point ofthe current are calculated. The most probable movementsare assigned to each object and added to the trajectory.Objects in the frame can move in different ways: theirtrajectories may intersect, they can disappear and ariseagain. To improve the accuracy of tracking, some man-ufacturers use the technology of sequence analysis andcontinuous post-processing of the results obtained. Wehave planned to use this approach in our system.
Another analogue of our system - GPS-trackers. Thesesystems work based on the definition of geolocation.To implement this solution, each employee must beequipped with a separate GPS tracker, the data fromwhich will be sent to the server at some interval. How-ever, this solution has a number of drawbacks:
1) The solution is not cost-effective, since it is nec-essary to purchase GPS trackers for all personnel.
2) We can’t exclude the situation in which the sellercan give his GPS-tracker to a partner to deceivethe system.
3) Such a solution is not universal. When identifyingsales consultants through the camera, it is possibleto expand the functionality, determine the level andtime of interaction of the seller with the buyer, andmuch more.
Also, analogs include systems for counting the numberof visitors on a video stream. These systems also havea number of shortcomings, the main one of which isthe impossibility of identifying sales consultants and thequality of their services.
IX. CONCLUSION.
In order to improve the technological process of de-tecting the seller’s consultant, it is possible to develop
279
additional functionality.To more accurately determine the seller’s consultant,
it is possible to analyze several elements of the uniformat once (for example, a yellow T-shirt and black pants).
Another factor that allows to detect the seller, canserve as a definition of behavior, characteristic for theseller-consultant. To solve this problem, you will needto create another neural network.
It is also possible to identify additional factors thatdetermine the quality of the seller’s work in the store,such as the presentation goods to customers who havenot stopped at the rack with the goods, but passing by.
In addition, it is planned to implement the ability toinstall "dead zones" in the program. This function allowsyou to set the “non-human” type for those objects thatthe Fast Pose Estimator neural network defines as people.
Thus, the developed software will make it possible toqualitatively improve the work of the sales assistant and,as a result, will lead to an improvement in the customerfocus of the business. Figure 4 shows an example of theprogram.
This work is a contin of the work, where the featuresand possibilities of determining the post-sense of itssemantic distinctive feature were considered [8] [9]. Thiswork was partially supported by RFBR (grants 17-29-07021, 18-47-340006, 18-47-342002, 18-07-00220, 19-07-00020).
Figure 4. An example of the program when recognizing an employeeof the store.
REFERENCES
[1] M.D.Khorunsjiy Metod kolichestvennoi otsenki tsvetovykh ra-zlichii v vospriyatii tsifrovykh izobrazhenii. [The method ofquantitative estimation of color differences in the perception ofdigital images.]. Nauchno-tekhnicheskii vestnik informatsionnykhtekhnologii, mekhaniki i optiki. [Scientific and technical heraldof information technologies, mechanics and optics], 2008
[2] O. Ulyanova Psihologicheskie osobennosti prodavcov-konsul’tantov setevogo marketinga. [Psychological featuresof network marketing sales consultants.]. 2013, Retrieved fromhttps://cyberleninka.ru/article/n/psihologicheskie-osobennosti-prodavtsov-konsultantov-setevogo-marketinga
[3] Cao Z., Simon T., Wei S.-E., Sheikh Y. Otsenka pozy v real’nomvremeni Multi-persony 2D s ispol’zovaniem polei blizosti. [Re-altime Multi-Person 2D Pose Estimation using Part AffinityFields.]. 2016, Retrieved from http://arxiv.org/abs/1611.08050
[4] Zhe Cao Otsenka 2D-otsenki v real’nom vremeni s is-pol’zovaniem otdel’nykh polei. [Realtime Multi-Person 2D PoseEstimation using Part Affinity Fields.]. 2017, Retrieved fromhttp://www.ri.cmu.edu/wp-content/uploads/2017/04/thesis.pdf
[5] U. Iqbal, J. Gall Otsenka individual’nosti cheloveka suchastiem mestnykh assotsiatsii. [Multi-person pose estimationwith local joint-to-person associations.]. 2016, Retrieved fromhttps://arxiv.org/pdf/1608.08526.pdf
[6] E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka,B. Schiele Bolee glubokii srez: bolee glubokaya, boleesil’naya i bolee bystraya model’ otsenki pozy dlya neskol’kikhchelovek. [Deepercut: A deeper, stronger, and faster multi-person pose estimation model.]. 2016, Retrieved fromhttps://arxiv.org/pdf/1605.03170.pdf
[7] Y. Osipove, D. Lavrov Primenenie klasternogo analizametodom k-srednih dlya klassifikacii tekstov nauchnojnapravlennosti. [Application of cluster analysis by k-meansmethod for classification of scientific texts.]. 2017, Retrievedfrom https://cyberleninka.ru/article/n/primenenie-klasternogo-analiza-metodom-k-srednih-dlya-klassifikatsii-tekstov-nauchnoy-napravlennosti.
[8] V.L.Rozaliev, Y.A.Orlova Opredelenie dvizhenii i pozy dlyaidentifikatsii emotsional’nykh reaktsii cheloveka. [Recognition ofgesture and poses for the definition of human emotions]. 11-ya Mezhdunarodnaya konferentsiya po raspoznavaniyu obrazov ianalizu izobrazhenii: novye informatsionnye tekhnologii (PRIA-11-2013), Samara, 23-28 sentyabrya 2013 g.: Trudy konferentsii[11th International Conference of Pattern Recognition and Im-age Analysis: New Information Technologies (PRIA-11-2013),Samara, September 23-28, 2013 : Conference Proceedings], 2013,vol. 2, pp. 713-716
[9] A.S.Bobkov, V.L.Rozaliev Fazzifikatsiya dannykh,opisyvayushchikh dvizhenie cheloveka. [Fuzzification of datadescribing the movement of a person]. Otkrytye semanticheskietekhnologii dlya proektirovaniya intellektual’nykh sistem(OSTIS-2011): mater. stazher. nauchno-tekhnich. konf. (Minsk,10-12 fevralya 2011 g.) [Open semantic technologies for thedesign of intelligent systems (OSTIS-2011) : mater. intern.scientific-techn. conf. (Minsk, Feb. 10-12. 2011)], 2011, pp.483-486/
[10] Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jiten-dra Malik Kompleksnoe vosstanovlenie formy cheloveka i pozy[End-to-end Recovery of Human Shape and Pose]. Retrievedfrom https://www.researchgate.net/publication/321902575_End-to-end_Recovery_of_Human_Shape_and_Pose?discoverMore=1
РАЗРАБОТКАМЕТОДА РАСПОЗНАВАНИЯПРОДАВЦОВ-КОНСУЛЬТАНТОВ НА ОСНОВЕНЕЙРОСЕТИ ДЛЯ ОПРЕДЕЛЕНИЯ ПОЗЫ И
ПОВЕДЕНИЯ
Розалиев В.Л., Алексеев А.В.,Ульев А.Д.,Орлова Ю.А.,
Петровский А.Б., Заболеева-Зотова А.В.
В статье представлен метод распознаванияпродавцов-консультантов на основе нейросети дляопределения позы и уточняющих алгоритмов, атакже рассмотрены методы контроля поведенияпродавца-консультанта и анализа его взаимодействияс покупателем. Проведен краткий обзор систем спохожими функциональными характеристиками.Представлено описание предлагаемой методики,показаны полученные результаты и пути улучшения
Received 10.01.19
280
Graph of TAPAZ-2 Semantic ClassifierAliaksandr Hardzei
Minsk State Linguistic UniversityMinsk, Belarus
Anna UdovichenkoLLC “SK Hynix Memory Solutions Eastern Europe”
Minsk, [email protected]
Abstract—The article discusses the algorithm for findingthe entrance to an arbitrary subject domain based on theTAPAZ-2 Semantic Classifier and the graph thus obtained.The formulas for the exact number of vertices in thegraph are derived depending on the number of elementsin the Paradigm of Actions and the restrictions imposedwhile constructing the graph. Various ways are proposed toreduce the number of vertices in order to adapt the power ofgraph combinatorics for automatic processing with moderntechnical means.
Keywords—paradigm of actions, semantic classifier, se-mantic ontology, vertex of the graph, degree of the vertex,artificial intelligence
I. INTRODUCTION
TAPAZ-2 is a tool for generating a model of theworld in a form suitable for Natural Language Processingin systems of Artificial Intelligence. The IntellectualKnowledge Base (IKB) built in a computer combinesthe Semantic Classifier – a final ordered (vector) set ofsemantic primitives (actions and roles of individs) andthe Semantic Ontology – an algorithm for generatingnew sense units based on the original set of primitives,presented in the form of the Semantic Classifier Graph[1].
An expert (intelligent) search system based on theSemantic Classifier may consist of an intelligent searchengine that selects and reviews content on a given topicfrom the Internet, and a dialog user interface that allowsthe system to process user requests and transform them inthe canonized text corresponding to the machine-readableModel of the World, and the user will confirm whetherthis conversion was performed correctly, and if not, thenoffer his own decoding through the Semantic Classifier.
II. DIMENSION OF THE SEMANTIC CLASSIFIERGRAPH
Let n be the number of actions in the initial generatinglist (the number of the first degree vertices) of theParadigm of Actions. The degree of the vertex S willbe called the number of vertices of the initial generatinglist directly or indirectly participating in the generationof this vertex. There can be several ways to generatean intermediate vertex from the same set of verticesof the first degree. The subgraph that specifies one ofthe possible ways to generate a vertex is a binary tree,
since exactly two ancestor vertices participate in thegeneration of each child vertex. Thus, the number ofways to generate an intermediate vertex of degree k isexpressed by the number of binary trees with the numberof leaves equal to k (the Catalan number) [2]. For k = 4,the number of such trees is 5, all possible configurationsare shown in Fig. 1.
Figure 1. All possible configurations of binary trees for k = 4
The number of k degree vertices is determined by theformula (1):
Nk = Akn × Ck (1)
where Akn – is the number of allocations from n
vertices with respect to k [3] (that is, the set of allthe vertices of the initial list participating in generatinga vertex) and Ck – is the Catalan number, which iscalculated by the formula (2) or (3):
Cn+1 =(2n)!
n!(n+ 1)(2)
Cn =(2(n− 1))!
(n− 1)!n!(3)
where n is the number of leaves of a binary tree [2].The maximum degree of a vertex in the Semantic
Classifier Graph is equal to the number of vertices inthe initial set (all vertices of the initial set participate ingenerating a vertex of degree n). Thus, in order to obtainthe number of vertices in the Graph, it is necessary tosum up the number of vertices of all degrees from 1 ton:
281
Sn =
n∑
k=1
Akn × Ck =
n∑
k=1
n!
(n− k)!× (2(k − 1))!
(k − 1)k!(4)
Python program code, calculating the function S(n)by the formula (4):
The Paradigm of Actions, which is used to buildthe Semantic Classifier Graph, contains 112 elements.Substituting this number into formula (4) as n, we get:
Sn ≈ 8, 2× 10245
At present, there are no devices whose computationalcapacities are capable of processing and storing a sim-ilar amount of information, therefore it is necessary toselect the most significant part of the Graph in order tosequentially generate, process and store this graph usingcomputer technology.
III. WAYS TO REDUCE VERTICES IN THE SEMANTICCLASSIFIER GRAPH
Now we can investigate the procedure for generatingnew actions similar to that described earlier, but in whichthe active action of degree k will be refined only by first-degree actions, i.e., a vertex of degree k and a vertex ofdegree 1 will always be involved in generating a vertexof degree k + 1. Such a graph is called a graph of thestandard form. Obviously, it will contain fewer verticesthan the Semantic Classifier Graph, since the graph of thestandard form is a subgraph of the Semantic ClassifierGraph. The subgraphs defining the method for generatingvertices of degree k have the following form (Fig. 2).
Figure 2. Subgraphs defining the generation of vertices in a graph ofthe standard form
At the same time, the order in which the clarificationoccurs is important: “warm by linking” is not the sameas “link by warming” [4]. We define the dimension
of the graph obtained in this way using the dynamicprogramming method [5] – the number of vertices ofdegree k will be expressed in terms of the number ofvertices of degree k − 1.
1) The number of vertices of degree 1 is n:
N1 = n (5)
2) The number of vertices of degree 2 is equal to thenumber of ways to form pairs, taking into accountthe order:
N2 = n× (n− 1) (6)
3) Fix a vertex of degree k − 1 (for k > 2) and wewill successively generate new vertices from thisvertex and all the remaining vertices of the firstdegree that are not active in its generation. Thenumber of ways to generate a vertex of degree kfrom one vertex of degree k−1, taking into accountthe order, is equal to:
nk = 2× (n− (k − 1)) = 2× (n− k + 1) (7)
where (n – k + 1) is the number of vertices of thefirst degree that are not involved in the generationof the fixed vertex of degree k − 1.Then the recurrence formula for the total number ofways to generate a vertex of degree k is as follows:
Nk = 2× (n− k + 1)×Nk−1 (8)
4) Sum up the number of vertices of all degrees from1 to n:
Sn =
n∑
k−1
Nk (9)
Python program code:
As a result of calculating the dimension of the graphfor n = 112, we obtain:
S112 ≈ 4, 2× 10215
Despite the fact that the number of vertices in sucha graph is less than in a full graph without restrictions,it is still extremely large and cannot be processed usingmodern computational tools.
282
In order to further reduce the dimension of the Seman-tic Classifier Graph, we impose an additional constraint:we fix one active vertex of the first degree (active action)and will consistently refine it with the rest (passive)actions. Then the subgraph of new actions with oneactive vertex will acquire the following properties:
• for any vertex of degree k, the first generatingancestor of degree k − 1 is a refined new actionof the active vertex, in other words, if you depicta method of generating a vertex of degree k as asubtree, then its first leaf will always be the activevertex;
• any vertex in the graph can be uniquely defined byan ordered list of first-degree vertices participatingin its generation, with the first vertex always beingthe active vertex and the degree of the vertex k beingthe length of this list.
Thus, the number of vertices in the graph will beequal to the number of ordered subsets of the first degreevertices’ set, excluding the active one (since it is alwaysfixed). This value is calculated by the recurrence formula[6]:
Tn = 1 + n× Tn−1 (10)
Then the total number of vertices in the graph (takinginto account all the vertices of the first degree) is:
Sn = n− 1 + Tn−1 (11)
Python program code:
As a result, the dimension of the graph for n = 112is:
S112 ≈ 4, 8× 10180
If we build a separate graph, alternately choosing eachof the 112 actions active, we get 112 graphs, containingin total the following number of vertices:
112∑
n=1
Sn = 5, 4× 10182
which is much less than the number of vertices inthe Semantic Classifier Graph of the same 112 actions.Although some of the possible meanings are lost, butsuch a construction can significantly reduce the numberof vertices in the graph.
The Paradigm of Actions contains 56 physical and 56information elements [7]. Let us analyze two separategraphs constructed on these two sets. Each of them willcontain the following number of vertices:
Table ITHE DIMENSION OF SEPARATE GRAPHS OF PHYSICAL OR
INFORMATION ACTIONS
SemanticClassifier
Graph
Graph ofthe Standard
Form
Graph withOne Active
VertexS56 1, 6× 10105 2, 1× 1091 3, 4× 1073
2× S56 3, 2× 10105 4, 2× 1091 3, 4× 1073
Despite the fact that the number of vertices is stillextremely high, the total number of vertices for graphsbuilt on half of the set of actions is significantly lessthan the number of vertices for the Semantic ClassifierGraph. Thus, due to partial losses of some variantsof meanings, it is possible to significantly reduce theamount of information to be processed.
Now we will try to reduce the number of vertices in theSemantic Classifier Graph from another angle: insteadof the restriction on the number of generated verticesof k degree, we introduce a restriction on the degree ofgenerated vertices. We will find the k degree at whichthe depth of detailing of the new actions is sufficient toachieve the required semantic power, but the number ofvertices in the graph remains within the limits allowing itto be processed and stored by modern computing means.
Table IITHE DIMENSION OF GRAPHS WITH LIMITED DEPTH OF DETAILING
Depth ofdetailingkmax
Semantic Classifier Graph Graph of the Standard Form
Sn Sn × 31 Sn Sn × 312 12544 388864 12544 3888643 2747584 85175104 2747584 851751044 748045984 23189425504 598986304 18568575424
Let us investigate the number of vertices in the Seman-tic Classifier Graph and in the graph constructed by themethod of sequential detail (Table 2). Already at the levelof kmax = 3 detailing, the number of generated vertices,multiplied by the number of k = 31 roles of individuals,exceeds 85 million (compare to the Dictionary of themodern Russian literary language in 17 volumes whichcontains 120,480 words, the declared volume of theLarge Academic Dictionary of the Russian languagewhich consists of 150,000 words, and the availableelectronic resources of the Institute of Linguistic Studiesof the Russian Academy of Sciences for 1.4 billion ofword usage which contain about 5 million Russian wordsof the XVIII – XXI centuries) [8].
At the level of kmax = 4 detailing, the number ofvertices of the Semantic Classifier Graph, multiplied bythe number of roles of individuals, exceeds 23 billion.
283
Since the generation of separate graphs for physical andinformational actions significantly reduces the number ofvalues obtained, this method of calculation will increasethe possible depth of detailing. These results are pre-sented in tab. 3, similar to tab. 2, only for a graph builton a set of 56 vertices of the first degree:
Table IIITHE DIMENSION OF THE GRAPHS OF PHYSICAL AND INFORMATION
PROCESSES WITH LIMITED DEPTH OF DETAILING
Depth ofdetailingkmax
Semantic Classifier Graph Graph of the Standard Form
Sn Sn × 31 Sn Sn × 312 3136 97216 3136 972163 335776 10409056 335776 104090564 44410576 1376727856 35595616 11034640965 6461701456 200312745136 3702618976 114781188256
At the depth of kmax = 4 detailing, the number ofvertices in the Semantic Classifier Graph, multiplied bythe number of roles of individuals, does not exceed twobillion. Thus, we get an additional level of detailing thatimproves the accuracy of calculating the subject domains.
IV. CONCLUSION
A search procedure of generating new actions throughthe set of first degree actions may be represented as agraph, a matrix, or a vector system. In the graph therelations of active first degree action and clarifying firstdegree action are represented as follows (Fig. 3):
Figure 3. TAPAZ-2 Semantic Classifier Graph
where: 1 – active first degree action; 2,3,4 – clarifyingfirst degree actions; 1-a , 1-b, 1-c – derivative seconddegree actions with 1-a as the active derivative seconddegree action; 1-a’ and 1-b’ – derivative third degreeactions with 1-a’ as the active derivative third degreeaction; 1-a" – the active derivative fourth degree action[4].
The total number of vertices in the TAPAZ-2 SemanticClassifier Graph is expressed by a number of 10245,which, of course, is too large not only for manual,but also automatic processing. Reducing the number ofvertices is achieved in three ways:
• restrictions on the generation of vertices;• division of the complete graph into two separate
subgraphs;• limiting the depth of actions’ detailing.
The combination of all three methods allows you toadjust the number of processed vertices, while, however,some of the meanings are lost. The question of limitingthe depth of detail without significant loss of meaningremains open for further research.
The second version of the Theory for AutomaticGeneration of Knowledge Architecture (TAPAZ-2) isone of the possible models for calculating semantics.Despite the fact that the model does not have analoguesin calculating of the subject domains, it does not claim tobe exclusive. Linguistic semantics is versatile and allowsdifferent ways of formalizing.
However, all methods, like Euclidean and non-Euclidean geometry, should be consistent and effectivein its problem solving, and those who argue with that,as Reichenbach aptly said, only “confuse a rigor of themethod with a limitation of a goal” [9].
REFERENCES
[1] A. Hardzei, Theory for Automatic Generation of KnowledgeArchitecture: TAPAZ-2, Minsk, 2017, 50p.
[2] R. P. Stanley, Catalan addendum. Available at: http://www-math.mit.edu/ rstan/ec/catadd.pdf.
[3] R. P. Stanley, Enumerative combinatorics. Cambridge, 2012,vol.1, 585p.
[4] A. Hardzei, Theory for Automatic Generation of KnowledgeArchitecture: TAPAZ-2. Minsk, 2017, 35p.
[5] T. H. Cormen, Ch. E. Leiserson, R. L. Rivest, Cl. Stein, Introduc-tion to Algorithms, 3d ed. Cambridge, Massachusetts, London,2009, 1292p.
[6] B. Schroeder, Ordered Sets: An Introduction, New York, 2003,391p.
[7] A. Hardzei, Theory for Automatic Generation of KnowledgeArchitecture: TAPAZ-2, Minsk, 2017, 34p.
[8] L. E. Kruglikova, El gran diccionario académico de lalenguarusa» como continuador de las tradiciones de la lexicografíaacadémica rusa. Cuadernos de Rusística Española, 2012, No.8.
[9] M. Reichenbach, Philosophie der Raum-Zeit-Lehre, De Gruyter,1928, 386 p. English translation: The Philosophy of Space andTime, New York, Dover Publications, 1958, 295p.
ГРАФ СЕМАНТИЧЕСКОГОКЛАССИФИКАТОРА ТАПАЗ-2А. Н. Гордей, А. М. Удовиченко
В статье рассматривается алгоритм поиска входа впроизвольную предметную область на базе семантиче-ского классификатора ТАПАЗ-2 и получаемый такимобразом граф. Выводятся формулы точного количе-ства вершин в графе в зависимости от количестваэлементов в таблице макропроцессов и ограничений,накладываемых при построении графа. Предлагаютсяразличные способы сокращения количества вершинграфа в целях адаптациимощности его комбинаторикидля автоматической обработки современными техни-ческими средствами.
Ключевые слова: таблица макропроцессов, семан-тический классификатор, семантическая онтология,вершина графа, степень вершины, искусственный ин-теллект.
Received 10.01.19284
Development of universal detection methodsfor identifying chronological or
pseudo-chronological order of occurrence ofterms in a given subject area
Ekaterina Filimonova, Sergey Soloviev, Irina PolyakovaLomonosov Moscow State University
Moscow, [email protected], [email protected], [email protected]
Abstract—This work seeks to develop universal de-tection methods for identifying chronological or pseudo-chronological order of occurrence of terms in a givensubject area. To solve the problem of reconstruction ofthe chronological order of words and terms, it is proposedto use three methods: the method of word formation,the method of dictionary use, and finally, the method ofhyponyms and hyperonyms.
The method of word formation can be divided intoseveral ways in relation to the problem: the prefixal method,the suffixal method, the prefixal-suffixal method, the non-suffixal method and the merging method. Prefixal methodof word formation forms a new word by adding a prefixto the base. The suffixal method of word formation formsnew words by adding a suffix to the base. Prefixal-suffixalmethod is based on the two methods of word formationdescribed above. The non-suffix method forms new wordsusing a zero suffix. The merging method forms new wordsby adding existing words. The method of using etymologicaldictionaries makes it possible to identify the exact sequenceof the terms according to the available accurate datacollected by such people as Max Vasmer.
Each method builds the order of words and terms asthey appear and is taken with a certain confidence factorof that order.
I. INTRODUCTION
It is known that different words in Russian languagecan have either direct or indirect connections.
In addition to the usual everyday words in the Russianlanguage is a special category of words, referred to asterms. Terms represent an area of special vocabularyof the language formed as a result of scientific andtechnological progress.
Terms are created by a person to be able to commu-nicate in various special areas. They should accuratelyreflect the results of people’s experience and practice.Terms should be concise, specific, precise, and unam-biguous. Terms can be formed by monosyllabic nouns,complex words, phrases, etc.
For example, monosyllabic nouns could be the wordsoil (’почва’), politics (’политика’), regions (’регио-
ны’), enterprises (’предприятия’), economy (’экономи-ка’).
For example, complex words could be the wordagriculture (’земледелие’), engineering (’машинострое-ние’), biosphere (’биосфера’), pricing (’ценообразова-ние’).
For example, phrases could be the word informa-tion security pricing (’информационная безопасность’),economic growth pricing (’экономический рост’).
Special terms of a particular subject area are usuallycollectively described in a Glossary, where each term isan object containing both name and definition.
For example, ’деньги - особый товар, выполняющийроль всеобщего эквивалента при обмене товаров, фор-ма стоимости всех других товаров. Деньги выполня-ют функции: меры стоимости, средства обращения,средства образования сокровищ, средства платежа имировых денег.’
Glossary is user-friendly, as it contains terms togetherwith their definitions, also including links. Over time,some terms become obsolete and go out of circulation.
For example, ’чеканка - получение рельефных изоб-ражений на листовом металле. Чеканка: является од-ним из древнейших видов художественной обработкиметалла; выполняется ударами особым молотком почеканам; ведется по поверхности металлического ли-ста, положенного на эластичную подложку из особойсмолы. Различают механизированную и ручную чекан-ку.’
Obviously, there is a need to rank the terms by thetime of appearance relative to each other.
II. METHODS
To implement the task of searching for the chrono-logical order of words and terms, we need to use amethod that allows us to determine the sequence of theirappearance relative to each other by two given terms.We assign to each method a degree of confidence in thecorrectness of its work. While choosing the final result,
285
we will give preference to the method with the greatestconfidence.
The following is the description of three methods foridentifying the order of appearance of terms.
The first method is based on comparing morphemicstructures of given terms. There is a system that al-lows dividing the input word by morphemes with highaccuracy. It is not limited to the scope of a particularsubject area, so the resulting morphemic structure of theterm allows application of the word formation rules ofthe Russian language, in which the greatest interest are:prefixal, suffixal, prefixal-suffixal, non-suffixal. Also, tothe above methods, another method of merging is added.
Prefixal method of word formation forms a new wordby adding a prefix to the base. In the discussed case,only nouns and adjectives are used as terms, so it isnecessary to determine if a certain term originated earlieror later, however it is not difficult. It is worth noting thatthe prefixal method of word formation does not cause ajump between parts of speech: a noun is obtained from anoun, an adjective is an adjective. For example, the worddemobilization (’демобилизация’) was formed from theword mobilization (’мобилизация’) prefixed way, or theword prediction (’предсказание’) is derived from legend(’сказание’).
Terms can consist of only two parts of speech-adjectiveand noun, terms that are verbs, as well as other parts ofspeech, are not expected to be found, so the scheme ofthe prefix method in relation to parts of speech is asfollows:
Figure 1. Scheme of the prefixal method in relation to nouns andadjectives.
This scheme does not contain such parts of speech asthe verb, adverb, and others, because in this paper theconsideration of these cases is not required.
The suffixal method of word formation forms newwords by adding a suffix to the base. This method differsfrom the previous one, because a noun, an adjective anda verb can be formed from a noun, while a verb and anadverb from an adjective. But since we are interested inthe formation of nouns and adjectives in this problem,let’s consider the cases when either a noun forms anoun or a noun forms an adjective. For example, theword market (’рыночная’) formed from the word market(’рынок’) suffixal way.
Terms can consist of only two parts of speech-adjectiveand noun, terms that are verbs and other parts of speech,
are not supposed to be found, so the scheme of thesuffixal method in relation to parts of speech is asfollows:
Figure 2. Scheme of the suffixal method in relation to nouns andadjectives.
This scheme does not contain such parts of speech asthe verb, adverb, and others, because in this paper theconsideration of these cases is not required.
Prefixal-suffixal method is based on the two methodsof word formation described above. In the case of usingnouns and adjectives with the help of word formationprefixal-suffixal way to get the same parts of speech(adjectives and nouns), just like in the suffixal andprefixal ways. An example of using this method can beseen in a couple of words: armour (’оружие’) and disarm(’обезоружить’).
The scheme of operation of the prefixal-suffixalmethod in relation to parts of speech is as follows:
Figure 3. Scheme of the prefixal-suffixal method in relation to nounsand adjectives.
This scheme does not contain such parts of speech asthe verb, adverb, and others, because in this paper theconsideration of these cases is not required.
The non-suffix method forms new words using a zerosuffix. Thus the zero suffix in the letter and in the speechis not expressed in any way. The non-suffix methodallows you to change part of speech. Thus, a noun can beformed from a verb, an adjective or a noun, an adjectivefrom a noun, an adjective and a verb. An example of thismethod of word formation is a pair of words: smoothsurface (’гладь’) and smooth (’гладкий’).
The non-suffix method, like prefixal, prefixal-suffixaland suffixal, very comfortable. It is easy to identify andrecognize, but if we know what word from what wasformed. But we are faced with the inverse problem-toidentify the order of an unknown method, if it can beapplied at all. There is a complexity especially in the caseof suffixal and non-suffix method. It is not always clear
286
what method is used. It is precisely such cases that giverise to uncertainty. In other words, we either guess withthe answer or not, and the probability of hitting about 50percent. For the study of such cases, this method gives aninaccurate result, which means that the confidence factorwill also not reach the highest.
The scheme of the non-suffixal method for parts ofspeech for our problem is as follows:
Figure 4. Scheme of the non-suffixal method in relation to nouns andadjectives.
This scheme does not contain such parts of speech asthe verb, adverb, and others, because in this paper theconsideration of these cases is not required.
The merging method forms new words by addingexisting words. In this case, between the words can beput a hyphen. In some cases, between the componentsof the final word can be put a connecting letter, such as’o’ or ’e’. Often merging words forms a new word byremoving the end (sometimes suffix) of the first wordand joining the second. Accordingly, part of the speechof the resulting word will be determined by part of thespeech of the second word.
As an example, the word ’agriculture’ (’земледелие’),which is formed by merging the basics of ’land’ (’земл’)and ’deeds’ (’дел’) with the addition of the letter ’e’between them.
The scheme of operation of the merging method inrelation to parts of speech for our problem is as follows:
Figure 5. Scheme of the merging method in relation to nouns andadjectives.
This scheme does not contain such parts of speech asthe verb, adverb, and others, because in this paper theconsideration of these cases is not required.
Certainly when it comes to the methods describedabove, using the first method can give rise to ambiguity,that is, it will definitely not be clear which term wasformed first, the first from the second or vice verse. Itis impossible to solve this ambiguity with the help ofthe first method, leading to potentially inaccurate results,and, therefore, reducing the confidence factor for thisparticular method. Inaccuracies can be resolved with thefollowing method.
Let us consider the method of identifying the orderof terms based on their use in etymological dictionaries.Currently, a large number of dictionaries exists, where forevery term there can be found another term, from whichthe first one was formed. This "other" term is spelledout explicitly, and finding it leads to a correct result. Themethod does not generate ambiguities, the only problemis that it may not give the desired result when either theterm itself or the etymologically original term are notdescribed in the dictionary. That is, the method with theoverall final result will be taken into account with a largeconfidence factor, unlike other methods.
At the moment, created etymological dictionaries,which include a huge amount of words. For each word,you can find information that describes the origin of theword. So, for example, for the term ’policy’ (’политик’)in etymological dictionaries it is possible to find the wordfrom which it is formed. Let us turn to the etymologicaldictionary of Max Vasmer translated into Russian. Inaddition to other information, we can highlight the mainrelated to our task. In relation to our example, we can getinformation that the term ’policy’ (’политик’) is formedfrom the word ’city’ (’город’). Or, for example, anotherexample, where the term ’society’ (’общество’) comesfrom the word ’general’ (’общий’). This informationgives a complete and error-free result, because the in-formation in the etymological dictionaries is reliable.
However, in etymological dictionaries is not alwaysfound the right word with the necessary information. Inthis case, you can apply the method of word formationusing the above methods, such as prefixal, prefixal-suffixal, suffixal and non-suffixal. A word close to thiscan also be found in the etymological dictionary.
Finally there is the method of identifying the order ofterms based on the allocation of generalization and quo-tient, better known as the problem of finding hyponymsand hyperonyms.
Hyponym (Greek. uπo-under, below + oνoµα - name)is a concept expressing a particular entity in relationto another, more general concept. Hyperonym (super) -a word with a broader meaning, expressing a general,generic concept, the name of the class (set) of objects(properties, features).
A hyperonym is the result of a logical generalizationoperation or, in a mathematical sense, a complement toa set.
287
Figure 6. Hyponyms and hyperonym.
If one of the terms is a hyponym, and the other isa hyperonym, and they have a common word, then thehyperonym appeared before the hyponym.
For example, for the pair of terms: art (’искусство’)and theatre arts (’театральное искусство’), the term artis the hyperonym and the term theatre arts – hyponym,which suggests that the term theatre arts came later ofthe term art. The complexity is represented in a situationwhere hyponym and hyperonym are not syntacticallysimilar. In this case, it is impossible to determine whichconcept arose earlier without any additional information.
Therefore, this method does not always produce acorrect result. Hence, the confidence factor will not behigh.
Of course, in the Russian language in addition to theabove difficulties, there are other ambiguities and obsta-cles to solving problems related to computer linguisticsin General. Consider one of these problems.
Polysemy, or polysemy of words occurs due to thefact that the language is a system limited in comparisonwith the infinite variety of reality, so that in the wordsof academician Vinogradov, " the Language is forced tocarry countless meanings in one or another headings ofthe basic concepts."
This problem could be another difficulty in achievingthis goal, but in our task the work is done with a specificGlossary. It is convenient not only because it contains alarge number of terms, but also because of the identifiedlinks and definitions.
Thus, there is no problem of ambiguity of understand-ing of terms. Connection with other concepts allows youto analyze the proximity of terms and the order of theirappearance.
III. CONCLUSION
Thus, several methods are proposed to solve the prob-lem of ranking terms by the time of their appearanceand to identify the chronological or pseudo-chronologicalorder of occurrence of terms in a given subject area.
REFERENCES
[1] Ye. Zemskaya, Sovremennyy russkiy yazyk. Slovoobrazovaniye:ucheb. posobiye, 3rd-ed. Russia, Moscow: Flinta.
[2] V. Nemchenko, Sovremennyy russkiy yazyk. Slovoobrazovaniye:Ucheb. posobiye dlya filol, Russia, Moscow: Vyssh. shk., 1984,p. 255.
[3] Ye. Ilina, YU. Dracheva, Sovremennyy russkiy yazyk:morfemika islovoobrazovaniye: uchebno-metodicheskoye posobiye, Vologda:VoGU, 2015, p. 68.
[4] I. Vasilyeva, D. Fedorov, Web-tekhnologii: uchebnoye posobiye,SPb.: SPbGEU, 2014, p. 67.
[5] L. Babenko, Lexicology of the Russian language. Textbook of theUral state Unversity named after M. Gorky, faculty of Philology,Ekaterinburg: Ural state University named after A. M. Gorky,philological faculty, 2008, p. 125.
[6] S. Barkhudarov, Lexical synonymy, Moscow: Nauka, 1967, p.180.
[7] Y. Apresyan, Lexical semantics. 2nd edition, revised and sup-plemented, Moscow: Languages of Russian culture; publishingcompany "Eastern literature" RAS, 1995, p. 472.
[8] I. Kobozeva, Linguistic semantics, M.: editorial URSS, 2000, p.352.
[9] V. Beloshapkova, modern Russian language. Second edition,corrected. and dop., M.: High school, 1989, p. 800.
[10] Er. Freeman, El. Freeman, Studying HTML, XHTML and CSS,Saint-Petersburg, 2012, p. 656.
[11] B. Hogan, HTML5 and CSS3. Web development according tostandards of new generation, Publishing house "Peter", 2011,p. 272.
РАЗРАБОТКА УНИВЕРСАЛЬНЫХМЕТОДОВВЫЯВЛЕНИЯ ХРОНОЛОГИЧЕСКОГО ИЛИПСЕВДОХРОНОЛОГИЧЕСКОГО ПОРЯДКА
ВОЗНИКНОВЕНИЯ ТЕРМИНОВ В ЗАДАННОЙПРЕДМЕТНОЙ ОБЛАСТИ
Филимонова Е. А., Соловьев С. Ю., Полякова И. Н.В работе ставится задача разработки универсальныхметодоввыявления хронологического или псевдохронологическогопорядка возникновения терминов в заданной предметнойобласти. Для решения задачи реконструкции хронологиче-ского порядка возникновения слов и терминов предлагаетсяиспользовать три метода: метод словообразования, методиспользования словарей, а также метод гипонимов и гипе-ронимов.
Метод словообразования можно разделить на несколь-ко способов применительно к поставленной задаче: при-ставочный способ, суффиксальный способ, приставочно-суффиксальный способ, бессуффиксный способ и спо-соб слияния. Приставочный способ словообразования фор-мирует новое слово добавлением приставки к основе.Суффиксальный способ словообразования формирует но-вые слова добавлением суффикса к основе. Приставочно-суффиксальный способ основан на двух описанных вышеспособах словообразования. Бессуффиксный способ форми-рует новые слова при помощи нулевого суффикса. Способслияния формирует новые слова сложением уже существую-щих слов. Метод использования этимологических словарейпозволяет по имеющимся точным данным, собранным таки-ми людьми, как Макс Фасмер, выявить точную последова-тельность возникновения терминов.
Каждый из методов строит порядок слов и терминов помере их появления и берется с определенным коэффициен-том уверенности этого порядка.
Received 10.01.19
288
Information Retrieval and Machine Translationin Solving the Task of Automatic Recognition
of Adopted Fragments of Text DocumentsYury Krapivin
Brest State Technical UniversityBrest, Belarus, 224000
Abstract—The solution for the task of automatic recog-nition of adopted fragments in multilingual informationenvironment with the cross-language functionality and focuson the detection of both explicit and implicit adoptions oftext fragments by means of the well-developed linguistictext analysis, that is based on the knowledge of naturallanguage, together with the existed effective informationretrieval and machine translation tools is proposed in thearticle.
I. INTRODUCTION
The term “information retrieval” was proposed byCalvin Mooers in the late forties of the previous centuryand denoted a package of measures with the aim toautomate the process of searching for information in un-structured text documents (data searching) and searchingfor documents (document retrieval). Information retrieval(IR) is usually treated as a process of searching andproviding the user with the information according torequest that represents his information need [1].
The main tasks of IR are:
• Classical retrieval, comprising automatic indexingof documents and users’ requests (according toMoore’s definition).
• Retrieval of documents matched to the input one,when the request is represented by some document-pattern used to search the texts similar to it incontent. This type of retrieval can lead up to moreprecise results in comparison with classical retrieval[2].
• Topical retrieval, which covers the topical filtrationof documents, taking in to account the tonality andstylistic nuance, and is applied to detect the textswith special or peculiar vernacular vocabulary. Thesystems, applicative to solve these tasks, are focusedon the specific universe of discourse that permits tocarry out deep retrieval on a certain theme.
• Clustering and classifying of documents are aux-iliary technologies of informational retrieval usedfor more effective representation of its results viaautomatic classes identification of input request (thetask of clustering) or via adding every detected
document to one of predefined categories (the taskof classifying).
A qualitative solution of all tasks, mentioned above,needs to involve quite effortful approaches of tex-tual information analysis, based on natural languageknowledge, usually enclosed in dictionaries, grammars,organizing rules of syntactic and semantic structuresfrom words and phrases, etc., which constitute linguisticknowledge bases. The latter, by-turn, are the foundationsof linguistic processors functioning – the most powerfulautomatic text processing tools, without which, at themoment, it cannot do a single automatic text processingtask [3].
A. The task of automatic recognition of adopted textfragments
An actual task of automatic recognition of adopted textfragments (plagiarism) immediately refers to the task ofretrieval of documents matched to the input one.
At present, there are some systems of adopted textfragments recognition (e.g. AntiPlagiat [4], Ephorus [5],WCopyFind [6], JPlag [7], Copyscape [8] etc.), whichare based on the algorithms, implemented according towidely-spread approaches: strings coincident, feature-quantification, information retrieval [9].
The approaches mentioned above are mostly focusedon solving the task of automatic recognition of adoptedtext fragments either from the point of view that takesinto consideration a relevance of proper text documentsaccording to a certain specified similarity measure, orthat takes into consideration explicitly undefined lexicaladoptions of fragments, which means the same textfragment that belongs to different text documents, as wellas minimal discrepancies may be allowed, for example,due to the usage of parenthesis, synonyms etc. Thosesolutions are focused on the recognition of lexical adop-tions of text fragments, which take into account simplemorphological transformations and synonymy relations,and don’t apply a well-developed linguistic analysis oftext documents, as well as don’t propose to take intoconsideration more complicated text modifications (for
289
example, the usage of voice synonyms and different syn-onymous constructions at a noun-phrases level, object-parametrical relations; paraphrase etc.,). They don’t pro-vide any task solution in the case of implicit (semantic)adoption that refers to the fragments of different textdocuments of the same meaning, expressed via variouschains of symbols, in relation to a specified system ofknowledge.
A huge number of text documents in different lan-guages from the Internet and full-text databases, onthe one hand, and the availability and simplification oftools for their processing, on the other, substantiallycomplicate a qualitative solution of the task of auto-matic recognition of adopted text fragments, due to theneed of identifying both text fragments of documents,represented in the language of an input document, andin other languages from the analyzed language set,translated to the language of an input document before-hand, in the analyzing document. All this is referredto as cross-language functionality. The quality of thetranslated equivalent is the most important thing, and theapproaches of recognition of adopted text fragments usedfurther will greatly depend on it.
The usage of machine translation systems with theobjective to translate one of the couple of analyzingdocuments beforehand to the language of the second oneseems to be the most natural. It’s needed to take intoconsideration the fact that these systems need to be mul-tilingual, and therefore, the concept of building such acomplex machine translation system should take into ac-count many characteristics of the problem, including thestate of the utilized natural languages distribution in rela-tion to the text search database. Thus, the existed appliedsolutions (for example, Promt [10], SysTran [11], Retrans[12], Belazar [13], SMP B/R [3], Yandex.Translate [14],Google.Translate [15] etc.) can be utilized for the ma-chine translation of text documents. They implement, asa rule, the following fundamental approaches: statisticalapproaches [16, 17], including example-based methods[18, 19], and linguistic approaches – rule-based methods[20, 21].
The first group of methods implies the usage of paral-lel text corpora as the basis for equivalence calculationsof lexical items in different languages and their statisti-cal characteristics, as well as certain translation modelconstruction. The second implies a translation modelconstruction according to the set of linguistic rules,which specify the necessary depth of text analysis, aswell as the admissible transformation of the grammaticalstructure of the input text into the equivalent structuresof the output one.
The performed analysis, as well as the experienceof application of machine translation tools, which areavailable in open access in the Internet, has revealedthat until recently their qualitative characteristics for
arbitrary texts were relatively low. The certain machinetranslation systems for “too close” natural languages (forexample, Russian and Belarusian) are the only exception.Therefore, recently the researchers have made someefforts to solve the designated problem. In the context ofoverall task, the alternative solutions, related to methods,which take into consideration syntactical characteristicsof natural languages [22] and utilize various dictionariesand thesauri [23, 24], as well as comparable and parallelcorpora [25, 26], are proposed.
In this context the choice of an acceptable task solutionprovides the implementation of cross-language function-ality in the system of automatic recognition of adoptedtext fragments and depends on the cardinality of naturallanguages set, the volume of analyzing documents, theavailable computational and informational resources, aswell as the temporal limitations. In case of insufficientquality, preference may be given to the informationretrieval approach and the machine translation of a doc-ument search profile (DSP), rather than text documents.
The analysis of well-known contemporary Internetsearch services (for example, Google [27], Yandex [28],Bing [29], Baidu [30], Yahoo [31], Mail.ru [32], AOL[33] etc.) has revealed, that it is sensible to focus onGoogle search engine, in spite of its peculiarities andimposed restrictions that are the most significant in termsof the problem to be solved. For example, according to[34, 35] (as of April 2017) the global marketing sharepercentage, in terms of the use of search engines heavilyfavours Google, with over 77%. It’s interesting to notethat Google’s large market share is still on the increase.Last year the market share for Google was 67%, soGoogle has taken another 10% of the market from itsrivals in just the past 12 months (“Fig. 1”).
Figure 1. The global marketing share of search engines.
The graph below highlights the usage of search en-gines in Belarus in past 12 months (“Fig. 2”) [36]. Duringthe survey period, it was found that Google accounted for69 %, Yandex.ru – 26,77 %, Mail.ru – 3,28 %, Bing –0,35 %, Yahoo! – 0,31 % and Other – 0,29 % of searchqueries in the country.
Huge volumes of search space and severe time limi-tations imposed on the response duration of the systemof automatic recognition of adopted text fragments re-quire preliminarily the quick and efficient minimization
290
Figure 2. Search engine market share of Belarus in the period ofDecember 2017 – December 2018.
of the search space and then start the recognition ofthe adopted text fragments. It’s necessary to take intoconsideration that we are talking about multilingual in-formation environment and cross-language functionality,in other words the search space must contain all relevantdocuments from the search space, regardless of languagerepresentation. And this, in turn, requires the use ofmachine translation functionality either in relation to textdocuments, or to their search profiles.
Thus, in [37] the method, based on bilingual dictio-naries of concepts, actions, attributes that provides anefficient solution for the task of automatic recognition ofadopted fragments even at a semantic level, it is proposedto use. The following steps are performed (“Fig. 3”):
• the key words are marked out from the input docu-ment and the query search profile is built (QSP);
• the QSP is translated into the remaining (n-1) lan-guages via a multilingual lexical database;
• the search for relevant documents;• the formation of the minimized search space for the
purpose of fast and effective selection of documentsbelonging to it;
• the search for the adopted fragments in the textdocuments from the minimized search space.
Figure 3. The flowchart of the minimization procedure of the searchspace.
A well-known TF*IDF method [38] was used toautomatically create a query search profile (the inputinformation for the search engine). This method formsthe set of key words marked out from the input documentautomatically, or, in other words, it forms a search profileof a proper text document. The analysis of search querieshas revealed a new downward trend of the number ofkeywords usage. One of the examples is the averagenumber of typed search terms during online search inthe United States as of January 2016. During that month,28.24 percent of all U.S. online search queries containedtwo keywords (“Fig. 4”) [39]. This trend is obviouslya consequence of the optimization of the used indexingmechanisms for both documents and user search queries.
Figure 4. Average number of search terms for online search queriesin the United States as of January 2016.
So, the procedure of searching for the relevant docu-ments is a mechanism of the search space minimizing.The list of keywords, achieved by TF*IDF method,can be improved via the usage of synonymy relations,which are defined with the help of checking of lists ofsynonyms in the MModWN database [40], as well asby the corrections of the weight coefficients, which takein consideration lexical units membership of the mostinformative lexico-grammatical, syntactical and semanticclasses (“Fig. 5”).
Figure 5. A fragment of multilingual lexical database MModWN.
For example, semantic super-subordinate relationsbetween the concepts with numbers 109459609 →
291
109421558 and 109421558 → 109421888 define therelationships between proper synonymic sets in differentlanguages.
Thereby, the machine translation of the query searchprofile is performed as follows:
• the search for the key words in the multilinguallexical database;
• the selection of the equivalent synonyms, includingthe synonyms in different languages.
CONCLUSION
An involvement of the well-developed linguistic textanalysis that is based on the knowledge of naturallanguage, together with the existed effective informationretrieval and machine translation tools provide a facilityof qualitative solving the task of automatic recognitionof adopted fragments in multilingual information envi-ronment with the cross-language functionality and focuson the detection of both explicit and implicit adoptionsof text.
REFERENCES
[1] Mamchich, A.A. Modeli i algoritmy informatsionnogo poiska v mno-goyazychnoi srede na osnove tematicheskikh i dinamicheskikh korpusovtekstov, dis. kand. tekh. nauk, Minsk, 2011. – 122 p.
[2] Poisk znanii v Internet. Available at: http://poiskbook.kiev.ua/pbs.html.(accessed 2018, Oct).
[3] Voronkov N.V. Metody, algoritmy i modeli sistem avtomaticheskogoreferirovanija tekstovyh dokumentov, dis. kand. teh. nauk, Minsk, 2007.165 p.
[4] AntiPlagiat. Available at: http://www.antiplagiat.ru/. (accessed 2017, Dec).[5] Ephorus. Available at: http://www.ephorus.com/. (accessed 2017, Dec).[6] WCopyfind. Available at: http://plagiarism.bloomfieldmedia.com/
wordpress/software/wcopyfind/. (accessed 2018, Nov).[7] JPlag. Available at: https://jplag.ipd.kit.edu/. (accessed 2019, Jan).[8] Copyscape Plagiarism Checker - Duplicate Content Detection Software.
Available at: http://copyscape.com /. (accessed 2019, Jan).[9] Krapivin, Yu.B. K zadache avtomaticheskogo raspoznavaniya vosproizve-
dennykh fragmentov tekstovykh dokumentov [To a task of automaticrecognition of reproduced fragments of the textual documents], VestnikBrGTU, 2009, vol. 59, no. 5, pp. 109–112.
[10] PROMT Translator. Available at: http://www.promt.com. (accessed 2018,Oct).
[11] SYSTRAN. Available at : http://www.systransoft.com. (accessed 2018,Oct).
[12] RETRANS. Available at: http://www.retrans.ru/. (accessed 2018, Apr).[13] Belazar. Available at: http://belazar.info. (accessed 2018, Oct).[14] Yandex.Translate. Available at: https://translate.yandex.by/help. (accessed
2018, Dec).[15] Google.Translate. Available at: https://translate.google.com/intl/en/about/.
(accessed 2018, Dec).[16] Brown, P.F. [et al.] A Statistical Approach to Machine Translation, Com-
putational Linguistics, 1990, vol. 16, no. 2, pp. 79–85.[17] Vogel, S. [et al.] Statistical Methods for Machine Translation,Verbmobil:
Foundations of Speech-to-Speech Translation, Springer Verlag: Berlin,2000, pp. 377–393.
[18] Beyond Translation Memories. Available at:http://www.eamt.org/events/summitVIII/papers/schaeler.pdf. (accessed2018, Feb).
[19] Turcato, D., Popowich, F. What is Example-Based Machine Transla-tion?,Recent Advances of EBMT / M. Carl, A. Way; ed.: M. Carl [et al.],Kluwer Academic Publishers, 2003, pp. 59–82.
[20] Kaji, H. An Efficient Method for Rule-Based Machine Translation, Compu-tational Linguistics: proceedings of the 12th Conference, Budapest, 22-27August 1988, ACL, 1988, pp. 824–829.
[21] Tamas, G., Gabor, H., Balazs, K. MetaMorpho TM: A Rule-Based Trans-lation Corpus, Language Resources and Evalutation: proceedings of the4th International Conference, Lisbon, 2004, pp. 339–342.
[22] Potthast, M. [et al.] Cross-language plagiarism detection, Language Re-sources and Evaluation, 2011. vol. 45, no. 1, pp. 45–62.
[23] Steinberger, R., Pouliquen. B., Hagman, J. Cross-lingual Document Sim-ilarity Calculation Using the Multilingual Thesaurus Eurovoc, Proceed-ings of the Third International Conference on Computational Linguisticsand Intelligent Text Processing, Computational Linguistics and IntelligentText Processing: Springer Berlin Heidelberg, Berlin-Heidelberg, 2002, pp.415–424.
[24] Pataki, M. A new approach for searching for translated plagiarism, Pro-ceedings of the 5th International. Plagiarism Conference, Newcastle-upon-Tyne, UK, 2012, pp. 49–64.
[25] Potthast, M. Cross-language plagiarism detection, Language Resources andEvaluation, 2011, vol. 45, no. 1, pp. 45–62.
[26] Muhr. M. External and Intrinsic Plagiarism Detection Using a Cross-Lingual Retrieval and Segmentation System - Lab Report for PAN at CLEF2010, Notebook Papers of CLEF 2010 LABs and Workshops, 2010, vol.1176, pp. 1–10.
[27] Google. Available at: https://www.google.com/. (accessed 2018, May).[28] Yandeks. Available at: https://www.yandex.ru/. (accessed 2018, May).[29] Bing. Available at: https://www.bing.com/. (accessed 2018, May).[30] Baidu. Available at: https://www.baidu.com/. (accessed 2018, Dec).[31] Yahoo. Available at: https://www.yahoo.com/. (accessed 2018, Dec).[32] Mail.ru. Available at: https://www.mail.ru/. (accessed 2018, May).[33] AOL. Available at: https://www.aol.com/. (accessed 2018, May).[34] Search Engine Statistics 2017. Available at:
https://www.airsassociation.org/airs-articles/item/19297-search-engine-statistics-2017. (accessed 2018, Dec).
[35] NetMarketShare. Available at: https://www.netmarketshare.com/search-engine-market-share.aspx?qprid=4&qpcustomd=0. (accessed 2018, Dec).
[36] StatCounter Global Stats. Available at: http://gs.statcounter.com/search-engine-market-share/all/belarus#monthly-201712-201812. (accessed 2019,Jan).
[37] Krapivin, Yu.B. Funktsional’nost’ cross-language v zadache avtomatich-eskogo raspoznavaniya semanticheski ekvivalentnykh fragmentov tek-stovykh dokumentov [Cross-language Functionality in the Problem of theAutomatic Identification of the Semantically Equivalent Fragments of theText Documents], Iskusstvennyi intellect [Artificial Intelligence], 2013, vol.62, no. 4, pp. 187–194.
[38] Robertson, S. Understanding Inverse Document Frequency: On TheoreticalArguments for IDF, Journal of Documentation, 2004, vol. 5, no 60, pp.503-520.
[39] U.S. online search query size 2017. Available at: http://www.statista.com.(accessed 2017, Dec).
[40] Krapivin, Yu.B. Lingvisticheskii analiz teksta v zadache avtomaticheskogoraspoznavaniya zaimstvovannykh fragmentov tekstovykh dokumentov [Thelinguistic analysis of text in a problem of automatic recognition of theborrowed fragments of text documents], Vestnik BrGTU, 2017, vol. 107,no 5, pp. 54–58.
ИНФОРМАЦИОННЫЙ ПОИСК ИМАШИННЫЙ ПЕРЕВОД В РЕШЕНИИ ЗАДАЧИ
АВТОМАТИЧЕСКОГО РАСПОЗНАВАНИЯЗАИМСТВОВАННЫХ ФРАГМЕНТОВ
ТЕКСТОВЫХ ДОКУМЕНТОВ
Крапивин Ю. Б.
В статье предложено решение задачи автоматиче-ского распознавания заимствованных фрагментов вмногоязычной информационной среде с функциональ-ностью cross-language и ориентацией на обнаруже-ние не только явных, но и неявных заимствованийфрагментов текста, на основании применения средствразвитого лингвистического анализа текста, опираю-щихся на знания о естественном языке в сочетаниис существующими эффективными инструментами ин-формационного поиска и машинного перевода.
Received 10.01.19
292
Recognition of Sarcastic Sentences in the Taskof Sentiment Analysis
Alexey Dolbin, Vladimir RozalievYulia Orlova, Sergey Fomenkov
Volgograd State Technical UniversityVolgograd, Russia
[email protected], [email protected]@gmail.com, [email protected]
Abstract—This article is devoted to the sarcasm recog-nition in the text written in a natural language. The maingoal is to increase the accuracy of sentiment analysis. Thesentiment level determination of a text that describes theappearance of a person was chosen as a domain areafor the experiment. At first, references to the personalityand elements that describes appearance from text aredetected using the method of latent semantic analysis. Thenext step is to evaluate the attitude to a person in textusing pre-labeled sentiment dictionary. At this stage, themethod of recognising sarcastic sentences that contains adescription of the appearance is used. The sentiment levelshould be re-evaluated in the person information model.The results of the experiment showed that the recognitionof sarcasm based on the morphological features of wordsand the frequency characteristics of the sentences doesnot effectively increase the accuracy of sentiment leveldetermination.
Keywords—sentiment analysis, named entity recognition,text mining
I. INTRODUCTION
Sentiment analysis of the text belongs to the categoryof information retrieval tasks. The importance of aneffective solution to this problem grows over time, sincethe amount of information that needs to be processedby semantic analysis systems is continuously increasing.At the moment there are quite effective methods forsentiment analysis of the text, but there are a numberof directions, the solution of which will make it possibleto achieve greater accuracy of correct recognition. Onesuch direction is the recognition of sarcasm. Sarcasm canbe classified as an implicit approach to the expression ofopposing emotions. However, even a person cannot al-ways determine reliably whether this phrase is a sarcasm.
The task of automating the definition of sarcasmitself is of little practical value. Typically, you need alimited application area to apply the sentiment analysis.And most often the development is carried out in thefollowing areas:
• Sentiment analysis of users reviews.• Analysis of comments posted on social media resources
[1].
The problem of recognizing sarcastic sentences in thetext in natural language was considered in the context
of searching for elements of a person’s appearance anddetermining the sentiment class. This named entity waschosen not by chance, since it is a quite complex task torecognize it with high accuracy due to a large numberof approaches to co-referencing through pronouns in thethird person.
The aim of this work was to examine modern methodsfor determining the author’s relationship to the describedperson by performing the sentiment analysis. The mostobvious area of application of the development, con-sidered in this article, is the analysis of comments onphotos in social networks. Using the methods of machinelearning, it is possible to construct a model that is able torecognize a positive or negative attitude to the appearanceof the person depicted in the photograph. The maincontribution of the authors of the article is the adaptationof existing methods of assessing the sentiment in thefield of recognition of a person’s appearance in the textin natural language [2].
II. INFORMATION MODEL
First of all, it was required to develop an informationmodel of a person’s appearance. This model must meetthe following requirements:
• Extensibility.• Visibility.• Completeness of the description.
A frame presentation of knowledge is perfectly suit-able for this description. Fig. 1 shows the final modelof a person’s appearance using the frame representationlanguage notation. There are the main components onwhich it is possible to compose a complete descriptionof a person’s appearance in the frame slots. Slots for themodel were compiled by the authors of the article. In thisfigure, "M" is the set of valid values for the descriptionelements of the appearance for each slot. The specialtyof the FRL notation is that it is permissible to joinspecial procedures-demons to it. The only procedure isthe determination of sentiment level with the subsequentresolution of sarcasm. It is worth noting that each non-empty slot must correspond to the sentences from whichthe facts were extracted for the frame. This is required
293
Figure 1. Simulation results for the network.
for further recognition of the presence of sarcasm in thetext [3].
A person is one the most difficult named entity torecognize in the text. It s not so difficutl to determineperson if text contains the name or surname of the entity.One common approach is to use contextual rules. Everyrule represents a standard regular expression, which isconstracted from the training sample as follows:
• Any mention of a person should be replaced with a specialword PERSON.
• If the word in a training sample is an element of a humanappearance, it should be set in its initial form.
• All other words should be replaced with its parts ofspeech.
• After processing the entire training sample, similar con-textual rules should be combined using special characters.The ‘?’ symbol means that this position can be omitted,the ‘+’ symbol means that the position can be repeatedone or more times in a row and the ‘|’ symbol representslogical “or”.
• Optionally, some phrases can be listed at the end ofthe training sample, which indicates that the sentenceexcludes the possibility of containing the entity.
Thus, the outputs are kind of regular expressions, whichare applied to the text to define a named entity with it.
The next step is to resolve the reference of pronounsin the third person. Reference resolution of pronouns inthe third form is one of the most common, but at thesame time the simplest case. This task can be consideredas a problem of binary classification. Therefore, it is agood opportunity to use support vector machines (SVM).The list of parameters for support vector machines whichwere used for training:
• Number of sentences between antecedent and anaphora.• Whether the antecedent is in the nominative.• Position of the anaphora in the sentence.• Position of the antecedent in the sentence.• Number of nouns and pronouns, which are located in
sentences with antecedent and anaphora.• Is the antecedent and anaphora case matches.• Is the antecedent and anaphora genus matches.• Is antecedent and anaphora both in a plural or singular
form [3].
To fill the frame, a method of latent semantic analysiswas used, or abbreviated LSA, as it has proved itselfin the field of machine learning. Methods that do not
use a pre-tagged training sample for the learning processshow a smaller effectiveness in terms of recognition. Themethod of latent semantic analysis can be characterizedas establishing the relationship between the vectors ofthe features of the analyzed documents to the words thatserve as the keys. Thus, to use the method of semanticanalysis of text in natural language, the slots of the frameshould be used as search keys [3].
The latent-semantic algorithm is as follows:• Create a list of all keywords that will be searched in the
text.• Create a frequency matrix A, in cells of which the count
of how many times does this word occur in the text.• Apply TF-IDF method on a frequency matrix to ensure
that results are relevant [4].• apply a singular matrix decomposition: algorithm divides
the transformed frequency matrix A into three compositematrices U, Vt and S according to (1)
A = U × S × V t (1)
• Matrix U contains the coordinates of keywords and Vt –coordinates of documents.
Singular value decomposition of the matrix allowsyou to get rid of unnecessary noise, which significantlyincreases the efficiency of the method. The number ofrows and columns that can be discarded before thesubsequent analysis can be selected experimentally. It isnow possible to obtain the nearest documents, which hasthe same semantic meaning as specified keyword, andthen fill in the frame slots.
III. VOCABULARY BASED SENTIMENT ANALYSIS
All approaches to determination of sentiment class aredivided into three main groups:
• Compilation of a sentiment vocabulary.• The use of various classifiers.• The use of compiled contextual rules.
A rule-based approach shows the most accurate results,but it requires very high costs and colossal linguis-tic work for compilation. The main drawback of thisapproach is that it is extremely difficult to composeuniversal rules that are suitable for all domains. Toachieve the most effective evaluation of the tonality, therules are compiled for a specific application area.
In this experiment, an approach based on the valencedictionary was applied, since it shows a fairly highpercentage of correct recognition. The task is greatlysimplified if there is a source for compiling a dictionaryof valences belonging to the domain under study. Suchdictionary was compiled on the basis of the corpus ofthe Russian language OpenCorpora. all the phrases thatare marked as "Qual" were chosen from this dictionary.Further, only those word forms that can be used fordescribing a person’s appearance have been filtered out.To simplify the task of sentiment analysis, it was decidedthat the valences would correspond to available sentimentclass. Table I shows an example of a sentiment dictionary.
294
Table IPART OF COMPILED SENTIMENT DICTIONARY
Keyword ValenceFriendly 2
Unfriendly -2Shy 0
Impartial 1Mean -1
For the study, the basic five sentiment classes werecompiled:
• Negative.• Strongly negative.• Positive.• Strongly positive.• Neutral.
To determine the sentiment, the naive Bayes methodwas used. This method has proved itself in the fieldof machine learning. A naïve Bayesian algorithm isa classification algorithm based on the Bayes theoremwith the assumption of independence of features. Theclassifier assumes that the presence of any feature in theclass is not related to the presence of any other attribute.Let the P(d|c) be the probability of finding a documentin all the documents of a given class. The basis of thenaive Bayesian classifier is the corresponding theorem(2). In (2) P(c) is the probability of certain documentcan be found among all data set and P(d) – probabilitythe document occurs throughout the whole corpus.
P (c|d) = P (d|c)× P (c)
P (d)(2)
Thus, the naive Bayes method is based on the problemof finding the maximum probability of a document be-longing to a certain sentiment class. Thus, the sentimentlevel for each key element of a person’s appearance canbe determined by (3) [5].
Pmax = argmax[P (c)
n∏
i=1
P (wi|c)]
(3)
Classification using naïve Bayes is easy and fast andrequires less training data. Also, it is better suited forclassification based on categories (sentiment analysiswith separate defined classes refers to such cases). How-ever, if there is some value of a category characteristic inthe data set that was not found in the training samples,then the model will assign a zero probability to this frameslot. Sentiment class for each key element of a person’sappearance can be determined by (3), where P(w|c) isprobability of occurrence of a certain term in a document.
Experimentally, it was found that the hierarchicalclassification gives better results than flat, because foreach classifier, you can find a set of features that allowsyou to improve results. However, it requires a lot of timeand effort for training and testing. Fig. 2 shows the finalclassifier based on the naive Bayes method.
Figure 2. The Hierarchical Structure of Sentiment Classes.
IV. APPROACH TO SARCASTIC SENTENCESDETERMINATION
The issue of sarcasm recognition in a sentence requiresthe training of another classifier. To solve this issue, themethod of k-nearest neighbors is used [6]. To classifyeach of the test sample objects, you must perform thefollowing steps sequentially:
• Calculate the distance to each of the training sampleobjects.
• Choose k training sample objects, the distance to whichis minimal.
• Class of the object being classified is the class most oftenencountered among the k nearest neighbors.
The following set of parameters for the vector ofsingularities was compiled:
• The presence of word forms, which are specific forsarcasm (such expressions include common words fromthe Internet slang).
• The presence of quotes in the text (if there are quotes, itis most likely that the text contains a certain degree ofirony).
• High frequency of punctuation.• The presence in the text of words that are most often used
in conjunction with sarcasm for a particular language,which are taken from training samples [7].
For this case, the weight is given as a function of thedistance to the nearest neighbors. In (4) d(x, x(i)) is afunction which determines the distance between elementsin a vector space. Equation (5) finally determines whetheror not the text being analyzed contains sarcasm, whereZi is a sum of weights for all of the available classes.If so, then the class of the slot must be changed to theopposite.
w(x(i)) = w(d(x, x(i))) (4)
C = argmaxZi (5)
Empirically, it was revealed that the classifier givesthe best efficiency in terms of accuracy if it analyzesthe nearest-neighbor number K equal to the number ofsentiment classes.
To obtain more plausible results, you should filterout the most frequent words in the model. This stepremoves unnecessary noise that could affect the finalresult of the study. In addition, before using the Knearest neighbors method, the volume of aggregatedsentiment information should be considered. In this studythe results for unigrams and trigrams are provided [8].
295
V. CONDUCTING THE EXPERIMENT
To decide whether a sentiment recognition effective-ness is better or worse using the method of sarcasmdetermination, a numerical metric is needed. For mostmodern algorithms based on machine learning, metricsof accuracy and completeness of search are used. Theaccuracy of the search determines the proportion ofdocuments that really belong to a given sentiment classacross all documents of this class. The completeness ofthe search determines the ratio of the found classifiersof documents belonging to this class to all documents inthe sample. Since in real practice of machine learningthe maximum accuracy and completeness of search areunattainable simultaneously, the analysis of results usingthe F-measure will be the most acceptable. The F-measure is calculated using (6).
F = 2Precision×Recall
Precision+Recall(6)
A training set consists of 500 samples was compiled:150 of them were marked as "containing sarcasm" and350 were marked as "not containing sarcasm". This ratiobetween classes was not chosen randomly, since thelikelihood of evaluating the sentiment class of the textas positive or negative is much higher than sarcastic.The experiment was conducted on a sample of 100 texts,which are supposed to be a description to the differentphotos with no more than 200 words in length andcontains only the information about person’s appearance.
Table IIEXPERIMENT RESULTS
Recall Precision FUnigrams, without sarcasm 0.80 0.82 0.810Trigrams, without sarcasm 0.85 0.84 0.844Unigrams, with sarcasm 0.86 0.68 0.760Trigrams, with sarcasm 0.87 0.77 0.820
As can be seen from the obtained results (table II),the method of sarcasm recognition in the text slightlylowers accuracy due to a relatively large number of falsepositives. It can be concluded that lexical features andpunctuation signs are not enough to train the classifier ata sufficient level. Most often, sentences have a complexstructure, which cannot be treated as a "bag of words"and requires the use of contextual syntactic rules [9].
CONCLUSION
As a result of the experiment, it can be concluded thatthe resolution of the task of recognizing sarcasm in a textcontaining a description of the appearance of a person cannotbe effectively resolved only using the methods of machinelearning with supervision. As a further study, it requires thedevelopment of contextual rules based on the syntactic structureof the text. At this stage, the F-measure estimation showed thatthe method slightly reduces effectiveness due to a relativelylarge number of false positives. It may be worth consideringa deeper approach to analyzing emotions, as suggested by the
authors [10]. There are alternative approaches to solving theproblem of determining the tonality class of the analyzed text.For example, the use of a neural network for text analysis cansignificantly expand the boundaries of tonality classes. This isachieved by applying a suitable output function, as a result ofwhich the output is the probability with which a text fragmentbelongs to each class. This work was partially supported byRFBR (grants 17-07-01601, 17-29-07021, 18-07-00220, 18-47-343007, 18-47-342002, 19-07-00020).
REFERENCES
[1] A. Ritter, S. Clark, Mausam and O. Etzioni, “Named entity recognition intweets: an experimental study”, Proceeding of the conference on empiricalmethods in natural language processing, Association for ComputationalLinguistics, pp. 1524–1534, July 2011.
[2] A. Dmitriev, A. Zaboleeva-Zotova, Y. Orlova and V. Rozaliev, “Automaticidentification of time and space categories in the natural language text,”Proceedings of the IADIS Internation Conference, pp. 23–25, October2013.
[3] A. Dolbin, V. Rozaliev, Y. Orlova, “Recognition of a person named entityfrom the text written in a natural language,” Journal of Physics: ConferenceSeries, vol. 803, 5 p., 2017.
[4] B. Gebre, M Zampieri, P. Wittenburg and T. Heskes, “Improving nativelanguage identification with tf-idf weighting,” 8th NAACL Workshop onInnovative Use of NLP for Building Educational Applications, pp. 216–223, 2013
[5] H. Shimodaira, “Text classification using naïve bayes,” Learning and DataNote, vol. 7, pp. 1–9, 2014.
[6] D. Davidov, O. Tsur, A. Rappoport, “Semi-supervised recognition of sar-castic sentences in Twitter and Amazon,” Computational Natural LanguageLearning, pp. 107–116, July 2010.
[7] P. Carvalho, L. Sarmento, M.J. Silva, E. Oliveira, “Clues for detectingirony in user-generated contents: oh...!! it’s "so easy",” 1st internationalCIKM workshop on Topic-sentiment analysis for mass opinion, pp. 53–56, November 2009.
[8] A. Reyes, P. Rosso, “Mining subjective knowledge from customer reviews:a specific case of irony detection,” 2nd workshop on computationalapproaches to subjectivity and sentiment analysis, pp. 118–124, June 2011.
[9] R. Gonzalez-Ibanez, S. Muresan, N. Wacholder, “Identifying sarcasm inTwitter: a closer look,” 49th Annual Meeting of the Association forComputational Linguistics, vol. 2, pp. 581–586, June 2011.
[10] L. Volkova, A. Kotov, E. Klyshinsky, N. Arinkin, “A Robot CommentingTexts in an Emotional Way,” Communications in Computer and Informa-tion Science, vol. 754, pp. 256–266, August 2017.
РАСПОЗНАВАНИЕ ПРЕДЛОЖЕНИЙСОДЕРЖАЩИХ САРКАЗМ В ЗАДАЧЕ АНАЛИЗА
ТОНАЛЬНОСТИ
Долбин А.В., Розалиев В.Л.,Орлова Ю.А., Фоменков С.А.
Данная статья посвящена распознаванию сарказма в тек-сте, написанном на естественном языке. Основная цель -повысить точность анализа тональности текстов. Определе-ние уровня тональности текста, описывающего внешностьчеловека, было выбрано в качестве предметной области дляэксперимента. На первом этапе в тексте распознаются лич-ности и элементы описания их внешнего вида при помощиметода латентно-семантического анализа. На следующемэтапе определяется отношение к внешнему виду человекас использованием размеченного словаря валентности. Наданном этапе используется метод распознавания саркасти-ческих предложений, которые содержат описание внешнеговида личностей. В результате чего уровень тональностипереоценивается в информационной модели внешнего видачеловека. Результаты эксперимента показали, что распозна-вание сарказма на основе морфологических особенностейслов и частотных характеристик предложений не позволяетэффективно повысить точность определения уровня тональ-ности.
Received 10.01.19
296
Software Model of Analysis and Visualizationof Emotional Intonation of the Spoken Phrases
Boris Lobanov, Vladimir ZhitkoThe United Institute of Informatics Problemsof National Academy of Sciences of Belarus
Minsk, [email protected], [email protected]
Abstract—The purpose of this work is to develop asoftware model that provides a variety of ways to analyzeand visualize the intonation of the main types of humanemotions. A set of basic emotions and their acousticalcorrelations in human speech are described. A databaseof emotional speech and a software model of analysis andvisualization of emotional intonation are presented. Thesoftware model created is based on suggested before acomputer training system named “IntonTrainer”.
Keywords—speech intonation, melodic portrait, intona-tion analysis, basic emotions, emotional intonation, softwaremodel
I. INTRODUCTION
It is well known that human speech conveys not onlylinguistic messages, but also emotional information. Inthe theories of emotion, the emotional states are oftenmapped into a two or three-dimensional space. Thetwo major dimensions consist of a valence dimension(pleasant–unpleasant) and an activity dimension (ac-tive–passive) [1]
Generally accepted that there exist a set of basic orfundamental emotions such as: Calm (neutrality) – Joy– Sadness – Anger – Fear – Surprise. Below for eachof the listed emotions a description of the characteristicpsychological state of the person is given.
Calmness (neutrality) – a serene, balanced state ofmind, no anxiety, doubts, excitement, worries . . .
Joy is a positive emotional state, connected with thepossibility of sufficiently fully satisfying an actual need.
Sadness – a negative emotional state associated withthe received information about the impossibility of meet-ing the most important vital needs
Anger is an emotional state, negative in sign, usuallyoccurring in the form of affect and caused by the suddenappearance of a serious obstacle
Fear is a negative emotional state that appears whena subject receives information about a real or imaginarydanger.
Surprise – an emotional reaction to sudden circum-stances
In the previous works of the authors [2], [3] a softwaresystem, called “IntonTrainer”, designed to train learnersin pronouncing a variety of intonation patterns of speech
was described. This work is devoted to the furtherdevelopment of the system in the direction of using itnot only as a means of Computer Assisted LanguageLearning (CALL), but also as a means of experimentalresearch of various functional aspects of intonation.
Intonation is widely recognized as an important as-pect of speech that provides both linguistic and socio-cultural information. Many people define the purposeof intonation to express the emotional side of speechas the most specific for intonation, although it is littlestudied in linguistics. Understanding how emotions areexpressed in speech is important not only for its ownsake but it’s also important for understanding how canwe know how much of the F0 variability reflecting anemotional content? For this reason, the development of aspecialized system that allows for detailed analysis andvisualization of emotional intonation is relevant.
II. ACOUSTIC CORRELATES OF EMOTIONS IN HUMANSPEECH
Emotions in human speech may vary according todifferent physical characteristics [4]. Several researchershave studied the acoustic correlates of emotions in theacoustic features of speech signals [5], [6]. Accordingto [7], there is considerable evidence for specific modelsof voice expression for different emotions. Emotions cancause changes in respiration, phonation and articulation,which, in turn, affect the acoustic characteristics of thesignal [8]. There is also much evidence in the acousticpatterns of vocal affect expression [9].
At present there is little systematic knowledge aboutthe details of acoustic patterns that describe specificemotions in human voice expressions. Typical acousticfeatures that are considered to be strongly involved inthis process include the following:
• the level, range and shape of the contour of thefundamental frequency (F0), which reflect the fre-quency of vibration of the speech signal and areperceived as the pitch;
• the level of vocal energy, which is perceived asvoice intensity, and distribution energy in the fre-quency spectrum that affects voice quality;
297
• formants that affect articulation;• speech speed.For example, some emotional states, such as anger,
fear, and happiness (or joy), are considered to have a highlevel of arousal [5]. They are characterized by a tensevoice with a higher speech speed, high F0 and a widepitch range. However, sadness (or quiet sadness) andboredom are similar with slower speech, lower energy,lower tone, reduced pitch range and variability of bothemotions [10]. There is an influence of voice emotionson the excitation of t nervous systems, primarily infrequency and time, and secondly, in loudness and pro-nunciation [4].
The common characters of acoustic features of theemotions based on the dimensional analysis with F0variance, intensity, and speech duration (between silenceperiods) were also discussed in [11]. Happiness, fear,shyness and sadness are quite even with F0 variance, andsurprise, anger, and dominance have strongly varying F0;for the intensity, anger, surprise, disgust, and dominancehave the highest value, and sadness and shyness areweakest; the longest duration occurs with happiness,disgust, and surprise, and shyness and sadness haverelatively longer pauses between utterances.
III. EMOTIONAL SPEECH DATABASE
In present study we use the Ryerson Audio-VisualDatabase of Emotional Speech and Song (RAVDESS)[12]. The “RAVDESS” is a validated multimodaldatabase of emotional speech and song. The database isgender balanced consisting of 24 professional actors, vo-calizing lexically-matched statements in a neutral NorthAmerican accent. Speech includes calm, happy, sad,angry, fearful, surprise, and disgust expressions, and songcontains calm, happy, sad, angry, and fearful emotions.Each expression is produced at two levels of emotionalintensity, with an additional neutral expression. All con-ditions are available in face-and-voice, face-only, andvoice-only formats. The set of 7356 recordings wereeach rated 10 times on emotional validity, intensity, andgenuineness. Ratings were provided by 247 individualswho were characteristic of untrained research participantsfrom North America. A further set of 72 participantsprovided test-retest data. High levels of emotional va-lidity and test-retest intrarater reliability were reported.Corrected accuracy and composite “goodness” measuresare presented to assist researchers in the selection ofstimuli. All recordings are made freely available undera Creative Commons license and can be downloaded athttps://doi.org/10.5281/zenodo.1188976.
IV. ANALYSIS AND VISUALIZATION OF EMOTIONALINTONATION
Software model that makes available of analysis andvisualization of emotional intonation is based on com-puter trainer system (see: https://intontrainer.by “Fig. 1”)
provides additional visual feedback, as well as a quanti-tative estimation of the correctness of speech intonationin the process of teaching various foreign languages [2],[3]. To create a system that allows for detailed analysisand visualization of emotional intonation we add to thesystem some new functions described below.
Figure 1. The initial window of the Application.
A. Extended parametric display of melodic intonationportraits
The standard graphic of the UMP obtained whenselecting in the Main Settings Window (see: “Fig. 2”) ofthe “IntonTrainer” system Show UMP and Show F0.In case if the option Show derivative F0 is selected, thenwe will get a joint image of the UMP and its derivativefor the phrase “Dogs are sitting by the door”, pronouncedwith neutral emotion (see “Fig. 3”).
Figure 2. The settings window of the Application.
In the Main settings window there is also one newdisplay option for UMP and its derivative. With anadditional selection of the Show phase plane mode,we obtain the map in coordinates “F0 - dF0”, shownin “Fig. 4”.
In “Fig. 5” and “Fig. 6” shows the results of display,analysis and comparison of the two the same spokenphrases with neutral and sadness emotions in the twomodes described above.
The extended parametric display of melodic portraitsof intonation by the joint analysis of the UMP and its
298
Figure 3. Joint display of the UMP and its derivative.
Figure 4. Joint display in the coordinates “F0 - dF0”.
derivative allows one to take into account more subtledifferences in the intonation of the reference and spokenphrases. Adding the ability to display the UMP and itsderivative in the coordinates “F0 - dF0” made it possibleto make a visual comparison of the intonation of thereference and the spoken phrases more vivid.
B. Automating the procedure for marking the analyzedsignals to voice regions
At the previous version there was the only possibilityof preliminary automatic marking of reference sound filesinto voice regions, namely: by selecting the operation
Figure 5. Results of displaying and comparing spoken phrases: UMPsand their derivatives.
Figure 6. Results of displaying and comparing spoken phrases in thecoordinates “F0 - dF0”.
Mark Out File in the advanced settings section -“Acoustic Speech Database”.
In new version of the “IntonTrainer”, there is anadditional possibility of direct automatic marking ofreference signals. To do this it is necessary to select inthe Main Settings Window the Auto Marking mode.Signal segmentation into voice regions is carried out onthe basis of information about the presence of periodicityin the signal while the signal amplitude is present atsufficiently high amplitude - A0 (t). The user in the
299
section Auto segmentation is also given the opportunityto independently specify the absolute threshold values -A0 limit (%) and relative - Relative A0 limit (%).
“Fig. 7” shows the UMP of the phrase ”Are yo+u fromGermany?” built in the presence of markup on P, N, T- regions, and in “Fig. 8” shows the trajectory of F0 inthe case when each of the voice regions of the phrase isassigned the same index N.
Figure 7. An example of displaying the trajectory F0 (UMP) by manualmarking on the P, N, T - regions.
Figure 8. An example of displaying the trajectory F0 by automaticallygiven the same index N for each of voiced regions.
V. CONCLUSION
The term Emotional Intelligence (EI) became widelyknown in 1995 with the publication of Goleman’s book:“Emotional Intelligence – Why it can matter more thanIQ?”. EI is the capability of individuals (or AI systems)to recognize their own emotions and those of othersdiscern between different feelings and label them ap-propriately, use emotional information to guide thinkingand behavior, and manage and/or adjust emotions toachieve one’s goal(s). This work is the first step towardsthe creation of software tools for an objective analysisof the physical components necessary for assessing andsimulating so-called emotional intelligence.
VI. ACKNOWLEDGEMENT
This paper was supported by a BRFFR grant(Ф17МС-039)
REFERENCES
[1] Scherer, K.R., Schorr, A., Johnstone, T., Appraisal Processes inEmotion: Theory, Methods, Research.Oxford University Press,New York and Oxford, 2001
[2] Lobanov B. On a Way to the Computer Aided Speech IntonationTraining / B. Lobanov, H. Karnevskaya and V. Zhitko // Proceed-ings of 19th International Conference on Speech and Computer.Hatfield, Hertfordshire, UK , Springer, 2017, pp. 582-592
[3] Lobanov, B. A Prototype of the Software System for Study,Training and Analysis of Speech Intonation / Lobanov, V. Zhitko,V. Zahariev // Speech and Computer: 20th International Confer-ence, SPECOM 2018, Leipzig, Germany, September 18–22, 2018,Proceedings – Springer, 2018. – P. 337-346.
[4] Picard, R., Affective Computing, MIT Press, 1997[5] Banse, R., Sherer, K.R., Acoustic profiles in vocal emotion
expression. Journal of Personality and Social Psychology 70 (3),1996, 614–636
[6] Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss,B., A Database of German Emotional Speech, Proceedings Inter-speech 2005, Lissabon, Portugal
[7] Banse, R., Sherer, K.R., Acoustic profiles in vocal emotionexpression. Journal of Personality and Social Psychology 70 (3),1996, 614–636
[8] Scherer, K. R., Vocal correlates of emotion, in A. Manstead &H. Wagner (Eds.), Handbook of psychophysiology: emotion andsocial behaviors (pp.165-197). London: Wiley, 1989
[9] Scherer, K.R., Kappas, A., 1988: Primate vocal expression ofaffective state, in D. Todt, P. Goedeking, & D. Symmes (Eds.),Primate vocal communication (pp. 171-194). Berlin: Springer
[10] Breazeal, C., 2001. Designing Social Robots, MIT Press, Cam-bridge, MA
[11] Abelin, A., Allwood, J., Cross-linguistic interpretation of emo-tional prosody. In: Proceedings of the ISCA Workshop on Speechand Emotion, 2000
[12] Livingstone SR, Russo FA (2018) The Ryerson Audio-VisualDatabase of Emotional Speech and Song (RAVDESS): Adynamic, multimodal set of facial and vocal expressionsin North American English. PLoS ONE 13(5): e0196391.https://doi.org/10.1371/journal.pone.0196391
ПРОГРАММНАЯМОДЕЛЬ АНАЛИЗА ИВИЗУАЛИЗАЦИИ ЭМОЦИОНАЛЬНОЙ
ИНТОНАЦИИ В УСТНОЙ РЕЧИ
Лобанов Б. М., Житко В. А.
Данная работа посвящена описанию программноймодели позволяющей проводить анализ и визуализа-цию интонации различных эмоций в устной речи. Опи-саны базовые эмоции и их акустические проявленияв человеческой речи. Представлен набор эмоциональ-ных фраз и их визуализация и анализ. Программнаямодель реализована на ранее предложенной обучаю-щей системе “IntonTrainer”.
Received 28.12.18
300
Linguaacoustic resourcesfor belarusian speech synthesizers
Zianouka E.The Center for belarusian culture, language and literature
National Academy of SciencesMinsk, Republic of [email protected]
Abstract—The article describes an algorithm for creatinga linguaacoustic data array for Belarusian text-to-speechsystems, which consists of 350 communicative-syntacticunits of a text corpus with coverage of all possible syn-tactic structures of statements and full punctuation of theBelarusian language. It also gives a reason for creating suchresources for synthesizers, depicts its constituents and theprinciple of material processing.
Keywords—computational linguistics, linguaacoustic re-sources, data array, text-to-speech synthesizers
I. INTRODUCTION
Today computer technology is used in almost allspheres of human activity. One important area of itsapplication is the development of text-to-speech systems(TTS), which automatically convert an electronic text tospeech. High-quality TTS have wide potential in variousfields of economy, science, culture, medicine, educationand others [1]. There are a huge number of synthesizersthat handle different languages. Speech synthesis andrecognition laboratory of UIIP NASB for the past 55years has been developing belarusian text-to-speech andrecognition systems. Today, stationary platform of syn-thesizer "Multiphone-4”, speech synthesizer for mobileplatforms and its internet-version are built up [2]. Themain feature of these developments, in particular an inter-net synthesizer, is an open free access following the linkhttp://www.corpus.by/tts3/ [3]. The functionalities of thesynthesizer are quite high and diverse, but there are somedrawbacks. Therefore, the relevance for a linguaacousticdata array is explained by the necessity of improving thequality of information processing, adding supplementaryfunctions and testing existing ones through linguisticresources for such systems in the Belarusian language[4].
II. THE RELEVANCY OF LINGUAACOUSTIC DATAARRAY
Modern computer technologies allow for collectingand analyzing linguaacoustic resources, building themin TTS through regular formal rules and algorithmsthat have been outlined by natural language. Underlinguaacoustic resources is meant a set of organizedin a certain way speech and language data which are
saved in machine storage media and are used in variousfields of practice (education, industry, economy, cultureand art, etc.) [5]. Large linguistic data files are usedfor creating and developing efficient systems of textand speech processing. In TTS under linguaacousticresources is understood an array of language data, whichanalyze and test the input text (see figure 1). The generalscheme of the speech synthesizer is as follows[6]: aninput text is primarily processed by the text proces-sor, in which word stress, letter-phonemic processing,its splitting into syntagmas, the choice of intonationtype for each syntagma are taken place. Then markedphonemic text is fed to the input of two processors:prosodic and phonetic. The phonetic processor generatespositional and combinatorial allophones of phonemes.The prosodic processor determines the current values ofthe amplitude and duration of sounds, as well as thefrequency of the fundamental tone. It also answers forthe right and emphatic intonation of voiced text (seefigure 2). Prosodic processor requires more rework inthe Belarusian text-to-speech synthesizers, then underlinguaacoustic data array is meant voiced marked datasetwith all the intonation signs, types and constructions.Therefore, the main concept in this paper is voiced datawith intonation marking.
Figure 1. The structure of tex-to-speech systems
III. CRITERIA FOR MATERIAL COLLECTION
Due to the fact that the linguaacoustic data array isbeing created for TTS to process text into emphaticspeech, we must collect the material which formal al-gorithm could analyze. Since Belarusian synthesizers(BTTS) don’t have syntactic parser, the machine will be
301
able to analyze an incoming text only by formal indi-cation. These are syntactic structure of the sentence andpunctuation. BTTS dataset should include sentences orexpressions with various schemes of syntactic structuresand punctuation. The syntactic structure of the sentencethe Belarusian literary language is divided into [7]:
1) Simple sentences:• one-member sentences;• two-member sentences.
2) Complex:• compound;• complex;• conjunctionless sentences;• mixed complex sentences with include differ-
ent types of complex sentences.Except for the syntactic structure of the Belarusian
language we are interesting in punctuation, as punc-tuation markers are formal indicators of the numberof syntagmas and accentual units and the conditionalindicators of intonation type in text. Therefore, we turnto the analysis of the following items [8]:
1) The communicative type of statement (narrative,question, imperative);
2) Punctuation used in this sentence (period, ques-tion mark, exclamation point, comma, semicolon,colon, dash, hyphen, parentheses, brackets, braces,apostrophe, quotation marks, and ellipsis);
3) Number of syntagmas in this statement.
Figure 2. The prosodic processor structure
IV. THE COMPOSITION OF LINGUAACOUSTIC DATAARRAY
The edition "“Bielaruskaja litaraturnaja spadcyna: an-talohija u 2 tamach” was shoosen as bulk material for
the data array. It equips belarusian masterpieces of XX -XXI cc. Their authors are A. Adamovich, G. Dolidovich,R. Borodulin, A. Vertinsky, J. Yanishchits, N. Gilewicz,A. Makayonok, P. Brovko et al [9]. As noted above,we have drawn attention to the main three points whilecollecting the data. Thus, 350 communicative-syntacticunits have been defined, 100 of which will make lin-guaacoustic database, while the remaining 250 will beused for testing BTTS after inserting and verificatinglinguaacoustic database as part of a speech synthesizer.With 100 units selected 50 are narrative (about three, fourstatements for each syntactic unit of the text, taking intoaccount all possible syntactic constructions), 50 occurin interrogative and imperative sentences. In the processof selecting interrogative sentences we have taken intoaccount w-questions, general questions, questions whichbegin with participle and the number of syntagmas.Imperative statements were collected on the basis ofemotional or intention of expression and the number ofsyntagmas [7].
Unfortunately, today there is no computerized softwarefor the determination of the accent structure. This processis manual. Collected material is marked in accordancewith the conventional signs, introduced by the author,such as stress (nuclear stress and secondary), boundaries(the border of phonetic syntagma and the border of pho-netic phrase),tones Tags (rising tone, falling tone, rising-falling tone, falling-rising tone; rising + neutral + fallingtone;rising - falling + neutral + rising - falling tone;falling-rising +neutral + falling-rising tone; intonationof incompleteness, understatement, which is understood(is used in the blank). A fragment of the markup textmaterial for a database is shown in Table 1. It transmitsthe type of statement, calls the sentence structure, thenumber of syntagmas, the author remarks, syntactic unitand its marked-up version.
The next step is experimental recording ofcommunicative-syntactic units with all intonationfeatures of the Belarusian literary language. For this,the program Sonic Sound Forge 11 was chosen for thefurther audio recording, its editing and marking. Sincethis is a test recording and audio signal processing,adhere to the basic recording conditions: 8000 Hz 16bit Mono on a simple or a built-in microphone system[10].
After recording, the regions are accentuared (nuclear(n), prenuclear (p) and postnuclear (t)) in a sound trackfor visual perception of graphic contour, the quality ofits recording. For this we use program [email protected] 3 illustrates the process of recording a narrativesentence Cioplaje pavietra pieralivalasia na so+ncy inSonic Sound Forge. Then the audio has been divided intoregions (nuclear (n), prenuclear (p) and postnuclear (t))(see. Figure 4). This procedure is necessary to assess thecorrectness of statement’s intonation marking to accent
302
Figure 3. An audio recording and marking regions of the sentenceCioplaje pavietra pieralivalasia na so+ncy in Sonic Sound Forge
units and sound quality which is wholesome for thefurther recording of full version by an expert. Thisprocedure is made in the software "Inton@trainer" [11].This Application is used as an instrument in a numberof scientific and practical studies, namely the study ofindividual, emotional and stylistic features of intonation.Comparative evaluation of speech intonation in normand pathology. Estimation of the intonational qualityof synthesized speech. Figure 4 notes tonal intonationcontour of Cioplaje pavietra pieralivalasia na so+ncy,where you can check the quality of sound, estimate thecorrect marking of regions (in the figure 4 nuclear regionis mentioned by vertical lines).
The above mentioned steps are performed for all 350units of the text material on the basis of which furtherlinguaacoustic database will be created. An updated tableof text material comprises a reference to the audio fileand displays the graphic contour of communicative-syntactic units (Table 2). According to this link, theuser will automatically follow a link into a folder withthe necessary material, which is stored in the cloud onGoogle drive of the author.
Figure 4. Graphic tonal countour of the sentence Cioplaje pavietrapieralivalasia na so+ncy in Intontrainer
V. CONCLUSION
The algorithm for compiling linguaacoustic data arrayis shown above. The development of linguaacousticresources for the belarusian text-to-speech synthesizermakes it possible to improve the quality of the system’sfunctionality, continuous testing, analyse input, interme-diate and final data for the development and improvementof the system, collect new linguistic resources. Linguaa-coustic data array will allow to solve various problems.In particular, it makes possible to voice text informationwith a clear and distinct speech, which is similar to thehuman.
REFERENCES
[1] Rechevye texnologii. programmy: sintez rechi [Speechtechnology. Programs: speech synthesis]. Avaliable at:http://speech - soft.ru/index.php?a=inf&inf=view&id%20razdel=1181017375 (acessed 15.03.2018).
[2] Hetsevich Yu. S. Alharytmy linhvistychnay apratsouki tekstaudlya sintezu maulennya na belaruskay i ruskay movakh: dys.kand. tekhn. navuk [Algorithms of linguistic processing for speechsynthesis in Belarusian and Russian languages. Cand. Diss. Minsk,2012. 198 p.
303
[3] Sintezatar maulennya pa tekstse [Text-to-speech synthesizer].Available at: http://corpus.by/TextToSpeechSynthesizer/ (accessed01.07.2017).
[4] Zianouka Ja.S. Modern technology in text to speech transforma-tion: text-to-speech synthesizers. Trudy BHTU [BSTU Proceed-ings]. Minsk, 2018, No 1 (207) Print and media technologies„pp.77-83.
[5] Proxorenko O.G. Linguistics, linguodidactics, linguoculturology:current issues and prospects of development. Materialy I respub-likanskoj nauchno-prakticheskoj konferencii s mezhdunarodnymuchastiem [Proceedings of the I Republican scientific-practicalconference with international participation]. Minsk, BSU Publ.center, 2017. 289 p.
[6] Lobanov B. M. Komp’yuternyy sintez i klonirovanie rechi [Com-puter synthesis and speech cloning]. Minsk, Belorusskaya naukaPubl., 2008. 357 p.
[7] Nikolaeva T.M. Frazovaya intonaciya slavyanskih yazykov[Phrasal intonation of Slavic languages]. Moskva, Nauka Publ.,1977. 278 p.
[8] Vigonnaya L.T. Intanacyja. Nacisk. Arfaepija [Intonation. Stress.Orthoepy]. Minsk, Nauka i tehnika Publ., 1991/ 215 p.
[9] Bielaruskaja litaraturnaja spadcyna : antalohija. U 2 kn. [Belarusianliterary heritage: an anthology in 2 books]/ P.M. Lapo. Minsk,Belorusskaya nauka Publ., 2011. 944 p.
[10] Zianouka Ja.S. Modern technologies for intonation constructionsprocessing. Lingvistika, lingvodidaktika, lingvokulturologiya: ak-tualnye voprosy i perspektivy razvitiya: materialy II mezhdunar-odnoj nauchno- prakticheskoj konferkncii [Proceedings of the IIRepublican scientific-practical conference with international par-ticipation]. Minsk, BSU Publ. center, 2018. pp. 36-43.
[11] Intontrainer. Available at: https://intontrainer.by/ (accessed12.03.2018).
[12] Dutoit T. An Introduction to text-to-speech synthesis. Norwell,MA, USA: Kluwer Academic Publishers, 1997. 420 p.
[13] Rusanova O.A. Modern technologies of oral speech syn-thesis. Available at: http://docplayer.ru/38896137-Sovremennye-tehnologii-sinteza-ustnoy-rechi.html (accessed 29.07.2017).
[14] Taylor P. Analysisand synthesis of intonation using the tiltmodel.J. Acoust. Soc. America, 2000, vol. 107, no. 3. pp. 1697–1714.
[15] Zakhar’ev V.A. Multi-voice text-to-speech synthesis for theconstruction of natural language interfaces of intellige9+ntsystems. Otkrytye semanticheskie tekhnologii proektirovaniyaintellektual’nykh sistem: materialy mezhdunarodnoy nauchno-tekhnicheskoy konferentsii [Proceedings of the International Scien-tific and Technical Conference “Open Semantic Technologies forIntelligent Systems”]. Minsk, 2017, chapter 1, pp. 167-170.
ЛИНГВОАККУСТИЧЕСКИЕ РЕСУРСЫ ДЛЯБЕЛОРУССКОЯЗЫЧНЫХ СИСТЕМ
СИНТЕЗА РЕЧИ
Зеновко Е.С.
В статье описывается алгоритм по созданию линг-воакустического массива данных для белорусскоязыч-ных систем синтеза речи, которая состоит из 350коммуникативно-синтаксических единиц текстовогокорпуса с охватом всех возможных синтаксическихструктур высказываний и полной пунктуацией бело-русского языка. Актуальность создания подобной базыданных обусловлена тем, что существующие системысинтеза речи на белорусском языке, несмотря навысокий уровень качества, далеки от совершенства.Одной из возможностей использования синтезирован-ной речи является осуществление современных ком-пьютерных технологий, позволяющих проводить сбори совершенствование лингвистических ресурсов дляподобных систем и улучшение их функциональностичерез формальные регулярные правила и алгоритмы,написанные на естественном языке. Разработка необ-ходимых лингваакустычных ресурсов для синтезато-ра речи открывает возможность улучшения качествафункциональности системы, постоянного тестирова-ния системы, анализа входных, промежуточных иитоговых данных для доработки и совершенствованиясистемы, сбора новых лингвистических ресурсов. Со-здание базы позволит решать разнообразные задачи.В частности, проводить озвучивание электронных тек-стов выразительной речью, подобной человеческой.
Received 29.12.18
304
Genetic algorithm of optimizing the size, staffand number of professional teams of
programmersAnatoly Prihozhy, Arseni Zhdanouski
Belarusian National Technical UniversityMinsk, Republic of Belarus
Abstract—This paper considers the problem of estab-lishing and optimizing the teams of programmers takinginto account their proficiency and the level of skills inprogramming technologies and tools. It proposes a methodof formalizing and evaluating the qualification of individ-ual programmers and entire teams of programmers, andproposes a genetic algorithm of optimizing the size, staffand number of the teams. Experimental results on a set ofprogrammers graduated from universities of Belarus showthe ability of the system to find the teams of programmers,which increase the overall qualification of the teams by30%. The obtained results prove the practical importanceof the model, genetic algorithm and software in the field oftechnologies and tools for the management of professionalteams of programmers.
Keywords—programmer; technology; tool; qualification;team of programmers; team size; team staff; optimization
I. INTRODUCTION
Agile technology [1] of flexible software development pro-vides requirements and finds solutions due to the joint effortsof development teams and customers. It supports adaptiveplanning, evolutionary development, continuous improvement,and rapid-flexible response to changes. Although many techno-logical environments use Agile, it requires further developmentfor distributed programming teams. Tools and processes areimportant, but it is more important to have competent peopleworking together effectively. Improper distribution of work caneliminate the connection of leading experts. Assigning a job toa team that is hard to find an appropriate expert increases thetotal cost of recruiting and performing work, making it difficultto select the right people for the distributed team.
The success of a distributed team of programmers in theimplementation of a large project strongly depends on theadequacy of the technologies and programming tools used, aswell as on the ability of effectively decomposing the projectinto parts. These parts should be distributed among those teamsof programmers who have the necessary knowledge and skillsto perform them. Work [2] formulates a problem of optimalpartitioning of the given set of programmers into teams. Itis a multifactorial and poorly structured problem. In [2], thefollowing factors were taken into account: the productivityof each programmer, the ability of pairs of programmers toincrease or decrease productivity in the process of collabo-ration, the increase in interfaces’ cost between programmerswhile increasing the number of programmers in one team.Evolutionary optimization methods [3] [4] [5] are capable ofsolving this kind of problems. Work [5] develops a genetic
algorithm for solving this problem. However, it does not takeinto account the level of programmers’ knowledge of thetechnologies and programming tools that are necessary for theproject. Work [6] proposes a model of evaluating proficiencyof programmers.
This article analyzes modern technologies and programmingtools and evaluates the level of proficiency of each of theprogrammers as well as the level of proficiency of entire teamwith respect to these technologies and tools. The size and thestaff of each of the teams is optimized.
This paper is organized as follows. Section II presents amethod of evaluating the proficiency of programmers and wholeteams of programmers in technologies and tools. Section IIIdescribes a genetic algorithm of optimizing the size, staff andnumber of teams. Section IV presents experimental results, andlast section concludes the paper.
II. PROFICIENCY OF A PROGRAMMER AND A TEAMOF PROGRAMMERS IN TECHNOLOGIES AND TOOLS
A. Rating of programming technologies and tools
Analysis of the research results RedMonk company[7] has provided on the popularity of programming lan-guages and tools, as well as the research results the IEEESpectrum organization [8] has provided on rankings ofprogramming languages allows the evaluation of ratingof various programming technologies. The technologyrating indicates its importance and breadth, as well asindicates the requirements and constraints at least onemember of a team of programmers must meet.
B. Proficiency of a programmer in technologies and tools
We use a survey method to solve the problem ofassessing the level of programmers’ knowledge of tech-nologies and tools. Each of the programmers interviewedis asked to fill out a questionnaire in which he/she indi-cates the level of proficiency in each of the technologies.The proficiency level is determined by a five-point scale:0 - lack of knowledge of technology; 0.25 - the minimumknowledge; 0.5 - intermediate skills; 0.75 - extendedtechnology proficiency experience; 1 - expert knowledgeand experience. We consider a programmer as expert,if he/she possesses theoretical expert knowledge, hasdeveloped at least two large projects and has worked for
305
at least two years with this technology. The programmerhas advanced skills if he/she meets two criteria of thethree ones, and has intermediate skills if meets only oneof the criteria.
C. Programmer qualification
The qualification of programmer p ∈ P with regard tothe level of knowledge / possession of technologies ofset T in relation to the maximum level of knowledge /possession can be evaluated with Equation (1).
Qualif(p) =
∑
t∈Trank(t)× factor(p, t)
MaxQualif(1)
where
factor(p, t) =
level(p, t) if level(p, t) ≤ RLp
t
0 otherwise(2)
andMaxQualif =
∑
t∈Trank(t) (3)
The value of rank(t) belonging to range [0,1] is therank of the technology. The value of level(p,t) of therange [0,1] is determined by the results of the survey ofthe programmers. The threshold value RLp
t of level(p,t)reduces the chances of admission to work on the projectfor programmers with an insufficiently high level ofqualification. This value is selected from the range [0, 1].If RLp
t∼= 0 then start-up programmers or programmers
who possess of only small part of the technologies mayparticipate in the project. If RLp
t∼= 1 then only experts
may participate in the project. As a result, the value ofQualif (p) is in the range [0,1]. According to Equation(1), programmers who possess of preferably high-ratedtechnologies are more highly qualified than programmerswho preferably possess of low-rated technologies.
D. Qualification of a team of programmers
Let the whole set P of programmers be divided intok teams, producing set G = g1, ..., gk. Programmersof team g constitute set Pg . The average qualificationof team g, including Ng programmers, is defined asan average value Qualif (p) of qualifications over allprogrammers p ∈ Pg:
Qualifavg(g) =
∑
p∈Pg
Qualif(p)
Ng(4)
In addition to the average qualification Qualifavg(g),the most important parameter that characterizes teamg is the best representative qualification Qualif best(g),which is determined with Equations (5) and (6):
Qualifbest(g) =
0, if ∃t, oblgt(t),mxl(g, t) < RLgt∑
t∈Trank(t)×mxl(g, t)
MaxQualif, otherwise
(5)
mxl(g, t) = maxp∈Pg
level(p, t) (6)
Equation (5) uses the values as follows:• oblgt(t) is a predicate that takes value true if technologyt is mandatory for a team (see Table 1);
• mxlevel(g, t) is the level of the best representative ofteam g in technology t;
• RLgt is a threshold value of the level of the bestrepresentative of team g in technology t.
According to Equation (5), qualificationQualif best(g) over the best representatives is equal tozero if there is at least one mandatory technology tfor the team, for which oblgt(t) is true, and the levelmxl(g, t) of qualification over the best representativesof team g is less than the threshold value of RLg
t .The explanation is this team is not capable of carryingout projects without highly qualified specialists in keytechnologies.
The weighted qualification Qualifw(g) of team g isestimated with Equation (7) as the sum of qualificationQualif best(g) over the best representatives with weightλ, and the average qualification Qualifavg(g) withweight 1− λ.
Qualifw(g) = λ×Qualifbest(g)+ (1−λ)×Qualifavg(g)(7)
The weighted qualification at 0 ≤ λ ≤ 1 can takeany value in the range [0,1]. The larger the value of λ,the larger the weight of the qualification over the bestrepresentatives, and vice verso, the lower the value ofλ, the larger the weight of the average qualification ofthe programmers in the team. The average qualificationreflects the skills of programmers. The qualification overthe best representatives show opportunities for the growthof skills of the team members who are guided and trainedby technology experts, who are members of the team andare exemplary.
It is obvious, the teams with low qualifications cannotbe recognized as workable, and the participation ofsuch teams in the project is unreasonable or at leastcontroversial. We can formalize the exclusion of theappearance of such teams with the concept of weightedthreshold qualification:
Qualif(g) =
Qualifw(g) ifQualifw(g) ≥ RQg
0 otherwise(8)
It is reasonable to recommend to choose the thresholdvalue RQg of qualification from the range from 0.5 to1.0, depending on the requirements of the project andthe technologies that are used for its development, aswell as depending on the expected quality of futuredesign results. Bellow, the qualification of a team of pro-grammers will be understood as the threshold weightedqualification.
Team g is called redundant by qualification if there is aprogrammer p ∈ g such that Qualif(g\p) ≥ Qualif(g).
306
In other words, the qualification of team g after removingprogrammer p from it should not be lower than thequalification before removing of the programmer. Thiscan happen when at least one of two following conditionsare met:• programmer p is not the only best representative of teamg on any technology t ∈ T ;
• qualification Qualif(p) of programmer p is lower thanthe average qualification Qualifavg(g) over team g.
If set G divides the set of programmers into teams,then we can estimate the overall qualification of theteams as a sum of the teams’ threshold weighted quali-fications:
Qualification(G) =∑
g∈GQualif(g) (9)
The overall qualification Qualification(G) takes val-ues in the range from k × RQg to k, where k is thenumber of teams.
III. OPTIMIZING THE SIZE, STAFF AND NUMBER OFTEAMS
A. Problem formulation
Let Ω be a set of feasible solutions, which is a set ofvarious partitions of set P of programmers into variousset G of teams. The main parameter of solution G isthe overall qualification Qualification(G) of all teams.Therefore, we formulate the objective function as:
maxG∈Ω
Qualification(G) (10)
We formulate the set of feasible solutions over asystem of constraints and describe them in the form of re-quirements for programmers and teams of programmersto be proficient in technologies and programming tools.Requirement1. It refers to the minimum threshold
qualification level RLpt of skills the programmer p
of set P has regarding technology t. The minimumqualification level requirement for all technologies isrepresented with vector RLp = RLp
1, ..., RLpm. Equation
(11) determines the actual level of Lpt .
Lpt = rank(t)× level(p, t) (11)
Requirement2. It refers to the minimum thresholdqualification level RLg
t the best representative of teamg ∈ G of programmers has regarding technology t.For all technologies, the minimum threshold level isrepresented with vector RLg = RLg
1, ..., RLgm. We
determine the actual qualification level Lgt of the best
representative of team g for technology t with equationas follows:
Lgt = rank(t)×max
p∈Pg
level(p, t) (12)
Requirement3. It refers to the minimum-thresholdweighted qualification RQg of each team g ∈ G ofprogrammers. The demanded minimum qualification is
the same for all teams. The actual weighted qualificationQualifw(g) of a team is estimated by Equation (7).
Set G typically includes a special team of unemployedprogrammers, who are included in a reserve team. Thenumber Nempl of programmers included in a workingteam can be computed with Equation (13).
Nempl =∑
g∈GNg (13)
The number Nres of programmers who are unem-ployed and included in the reserve team can be computedwith Equation (14).
Nres = |P | −Nempl (14)
The average weighted threshold qualificationQualifavg over all teams can be evaluated withEquation (15).
Qualifavg =
∑
g∈GQualif(g)
|G| (15)
B. Genetic algorithm for solving the optimization prob-lem
The genetic algorithm (GA) implements a randomprocess of evolution of a population of chromosomes inorder to find the best partitioning of the set of program-mers into developers teams. We build the chromosomeas a vector of genes that correspond to the programmers.The gene value is the team number that includes the pro-grammer. The fitness function is the overall qualificationQualification(G) of the programmers of teams G.
The genetic operation of crossing two chromosomesrecombines their gen-parts and moves programmers fromone team to other team in two resulting offsprings. Thiscrossover operation can yield an offspring that does notrefer to a team of programmers, thus reducing the numberof teams. Such a situation may require re-enumeratingthe teams and introducing facilities, which can extendthe set of teams.
The genetic mutation operation randomly chooses oneor more programmers and transfers them to other teams.The selection operation selects parents according to therule of roulette wheel in order to perform crossingand mutation operations and to select chromosomesfor producing the next generation of chromosomes andfor updating the population. The chromosome with thehighest value of the fitness function is a solution of theoptimization problem.
IV. EXPERIMENTAL RESULTS
A. Rating of programming technologies and tools
Table I describes a set T of 16 basic technologies andtools, and reports the rating of each of them [7, 8]. Byappointment, the whole set of technologies is dividedinto 6 subsets. Version control and project managementsystems include Git, Tortoise SVN, VJR and TFS with
307
rating 0.3 each. The development environments includeVisual Studio and Eclipse, the rating of both is 0.6.Oracle SQL (rating 0.5) and Microsoft SQL Server(rating 0.6) represent database management systems.Programming languages are Java, C#, Visual Basic, C++,Java Script and XSL with rating of 1.0, 0.9, 0.7, 0.9,0.8 and 0.6 respectively. Windows and Linux representoperating systems with rating of 0.6 and 0.5 respectively.The high rating of a technology shows the groundlessnessof the creation of teams of programmers, in which thereis no any expert on this technology.
Table IKEY TECHNOLOGIES AND PROGRAMMING TOOLS
No Name Code Rating Oblgt1 Git VGT 0.3 no2 Tortoise SVN VTS 0.3 no3 TFS VTF 0.3 no4 Jira VJR 0.3 no5 Visual Studio DVS 0.6 yes6 Eclipse DEC 0.6 yes7 Oracle SQL OBM 0.5 no8 Microsoft SQL Server DBM 0.6 no9 Java LJ 1.0 yes10 C# LC# 0.9 yes11 Visual Basic LVB 0.7 no12 C++ LCP 0.9 yes13 Java script LJS 0.8 yes14 XSL LXS 0.6 no15 Windows OSW 0.6 yes16 Linux OSL 0.5 yes
B. Proficiency of programmers in technologies and tools
Fig.1 shows results of a survey of 24 programmers(set P ) with higher education, who graduated fromuniversities of Belarus and work at programming compa-nies. The rows correspond to the programmers, and thecolumns correspond to the technologies of Table I. Atthe intersection of row p and column t, the rectangle’sheight indicates one of the five levels 0, 0.25, 0.5, 0.75and 1 of possession by programmer p of technology t.The absence of a rectangle means zero level. Rows witha large total area of rectangles indicate highly qualifiedprogrammers. Columns with a large area of rectanglesindicate highly demanded and widely used technologies.
Figure 2 shows the level of importance of each ofthe 16 technologies and programming tools for 24 pro-grammers in relation to the highest possible level withouttaking into account the technology rating. The Windowsoperating system (code OSW) has the highest level of0.72. In second place is the version control system /project management system TFS (code VTF) with thelevel of 0.63.
Figure 3 shows the level of possession of technologiesand programming tools by 24 programmers, taking intoaccount the technology rating (Table 1). The Java pro-gramming language (code LJ) has the highest level of0.98. The Windows operating system (code OSW) withthe level of 0.73 has moved to the second place.
Figure 1. Proficiency of 24 programmers in 16 technologies
Figure 2. The level of possession of 16 technologies by 24 program-mers without taking into account the rating of technologies.
308
Fig. 4 shows the qualification of each of the 24 pro-grammers in terms of possession of all 16 programmingtechnologies without taking into account the technologyrating (in this case the rank (t) value of 1 for all t andthe MaxQualif value of 16 is taken). Figure 5 showsthe qualification of programmers on all programmingtechnologies, taking into account the rating of technolo-gies (in this case, the value of rank (t) is taken fromTable 1 and the MaxQualif value is 9.5). Note thatthe consideration of technology rating (Fig. 5) led tothe fact that programmer 8 became more qualified thanprogrammer 9 (in Fig.4, programmer 9 is more qualifiedthan 8). The same concerns programmers 22 and 23.
Figure 3. The level of possesion of 16 technologies by 24 programmerstaking into account the rating of technologies.
Figure 4. Qualifications of 24 programmers in all technologies withouttaking into account the technology rating.
C. Teams optimization constraints
We have developed a computer program that imple-ments the proposed genetic algorithm, and have used thisprogram for carrying out computational experiments onoptimizing the distribution of programmers on a set ofteams. The requirements and constraints are as follows:• the one-programmer qualification on each of the 16 tech-
nologies that are listed in table I must not be lower than
Figure 5. Qualifications of 24 programmers in all technologies takinginto account the technology rating.
RLp = 0.1, 0, 0.1, 0, 0.3, 0.25, 0, 0.15, 0.6, 0, 0, 0.25,0, 0.2, 0.4, 0.2;
• the one-team-of-programmers qualification on each of the16 technologies must not be lower than RLg = 0.2, 0, 0.2,0, 0.4, 0.3, 0, 0.3, 0.75, 0.4, 0, 0.5, 0, 0.3, 0.5.0.25;
• the threshold weighted qualification of one team of pro-grammers on all technologies must not be lower thanRQg taking value from the range 0.40 ... 0.75;
• the weight λ of one team qualification over the bestrepresentatives is equal to 0.7, and the weight 1 − λ ofone team average qualification is equal to 0.3.
D. Experimental results
Table II reports experimental results that are obtainedwith the genetic algorithm while optimizing the size,staff and number of teams composed of 24 programmers(Fig.1), using the 16 key programming technologies andtools (Table I). Previous Section describes the require-ments and constraints on teams establishing, which areassociated with a project the teams will work on. Varyingthe value of RQg in the range from 0.40 to 0.75 withstep 0.05, GA has given 8 solutions listed in TableII. The increase in the proficiency level RQg of oneteam decreases monotonically the number |G| of teamsfrom 9 down to 2. Moreover, it decreases the numberNempl of programmers involved in these teams. Thequalification Qualifavg of one team grows from 0.602to 0.785, although the overall Qualification(G) of allteams falls rapidly from 5.42 down to 1.57 due to manyprogrammers appear not involved in the project and areincluded in the reserve team. The staff of all teams isgiven in Table III.
Figure 6 shows the dynamics the genetic algorithmyields in the process of evolution of the chromosomepopulation. The total weighted threshold qualificationfor all teams has grown (Fig. 6) from 3.5 to 5.04over 25 generations. Growth is about 30%. During thistime, the number of teams that meet all qualificationrequirements has increased from 5 to 8, while the numberof programmers included in these teams has increasedfrom 15 to 23 . The number of teams that do not meet
309
Table IIEXPERIMENTAL RESULTS ON OPTIMIZATION OF PROGRAMMING
TEAMS WITH GENETIC ALGORITHM
Run RQg |G| Nempl Qualifavg Qualification(G)1 0.40 9 22 0.602 5.422 0.45 8 23 0.631 5.053 0.50 8 23 0.630 5.044 0.55 8 21 0.626 5.015 0.60 6 22 0.683 4.106 0.65 5 16 0.696 3.487 0.70 3 13 0.757 2.278 0.75 2 6 0.785 1.57
the requirements has ranged from 1 to 2, and the numberof programmers who are included in the reserve teamhas ranged from 9 to 1. As a result, the evolutionaryprocess has been accompanied by a steady increase inthe total threshold weighted qualification of the set ofteams represented by the best chromosome.
Table IIISTAFF OF WORKING AND RESERVE TEAMS
Run Working teams Reserve teamg1=6,7,15,17,24, g2=11,20,23, 4,9
1 g3=1,3,5,14, g4=8,19,21, g5=18,g6=16, g7=22, g8=2,12, g9=10,13
g1=8,13,15,21,24, g2=2,5,6,9,14, 202 g3=3,4,7, g4=12,22, g5=10,11,
g6=16,19, g7=1,17,23, g8=18g1=4,11,17,20, g2=3,5,14,19, 10
3 g3=7,9,13,22, g4=1,12,15,24,g5=2,6,21, g6=18, g7=8,23, g8=16
g1=3,4,6,9,11,17, g2=5,19,21,24, 2,20,234 g3=10,14, g4=8,12,15, g5=1,7,
g6=18, g7=16, g8=13,22g1=2,4,7,9,14,16, 19,20
5 g2=3,6,10,13,17,21,24, g3=12,22,g4=1,5,8,23, g5=11,15, g6=18g1=1,2,4,5,8,14,15, g2=12,16,20, 3,6,7,9,13,
6 g3=10,11, g4=19,22,23, g5=18 17,21,24g1=2,9,10,11,13,14,22,23, 1,3,4,5,6,8,
7 g2=7,12,15,16, g3=18 17,19,20,21,24g1=7,10,11,12,15, g2=18 1,2,3,4,5,6,8,
8 9,13,14,16,17,19,20,21,22,
23,24
CONCLUSION
This paper has presented a method of assessing the qual-ifications of development teams that takes into account theirknowledge and skills in programming technologies and tools.We have developed a genetic algorithm to optimize the size,staff and number of teams, which maximizes the overall qualifi-cation of the teams and minimizes the number of programmersnot involved in them. Further research will focus on theintegration of the qualification aspects and the performancesof programmers and teams of programmers, which participatein executing big projects.
REFERENCES
[1] Joshi, S. Agile Development - Working with Agile in a Distributed TeamEnvironment / S. Joshi // MSDN Magazine, 2012, Vol.27, No.1, pp.1-6.
[2] Prihozhy, A. Lecture notes on “Modeling and Optimization for EngineeringSystems Design” / A. Prihozhy // BNTU, Software for Computers andAutomated Systems Dpt., 2013, pp. 58-69.
Figure 6. Weighted total qualification of all teams of programmers vs.generation number in GA.
[3] Barricelli, N.A. Symbio genetic evolution processes realized by artificialmethods / N.A. Barricelli // Methodos, 1957, pp. 143-182.
[4] Muller, J.P., Rao, A.S., Singh, M.P. A-Teams: An Agent Architecturefor Optimization and Decision-Support, Proceedings 5th InternationalWorkshop, ATAL’98 Paris, France, July 4-7, 1998, pp. 261-276.
[5] Prihozhy, A.A. Evolutionary Method of Software Teams Optimization forReducing Time and Resources of Project Execution / A. Prihozhy, A.Zhdanouski // Proc. Conf. “Information Technologies in Engineering andBusiness”, Minsk, RIHS, 2016, pp.16-20.
[6] Prihozhy A., Method of qualification estimation and optimization ofprofessional teams of programmers / A. Prihozhy, A. Zhdanouski // Systemanalysis and applied information science, No 2, 2018, pp. 4-12.
[7] Red Monk, [electronics resource] / the developer-focused industry an-alyst firm Red Monk. – Access mode: http://redmonk.com/sogrady/2016/07/20/language-rankings-6-16/. – Date of access: 26.02.2017.
[8] Cass, S. The 2016 Top Programming Languages [electronics resource]/ IEEE Spectrum, 2016. −− Access mode: http://spectrum.ieee.org/computing/software/the-2016-top-programming-languages. – Date of ac-cess: 26.02.2017.
ГЕНЕТИЧЕСКИЙ АЛГОРИТМОПТИМИЗАЦИИ ЧИСЛЕННОСТИ,
ПЕРСОНАЛА И КОЛИЧЕСТВАПРОФЕССИОНАЛЬНЫХ КОМАНД
ПРОГРАММИСТОВПрихожий А.А., Ждановский А.М.
В статье рассматривается проблема созданияи опти-мизации команд программистов с учетом их квалифи-кации и уровня навыков в технологиях и инструментахпрограммирования. Она предлагает метод формализа-ции и оценки квалификации отдельных программистови целых команд программистов, а также предлага-ет генетический алгоритм оптимизации численности,персонала и количества команд. Экспериментальныерезультаты на множестве программистов, окончившихвузы Беларуси, показывают способность системы на-ходить команды программистов, которые повышаютсуммарную квалификацию коллектива разработчиковна 30%.Полученныерезультатыподтверждаютпракти-ческую значимость модели, генетического алгоритмаи программного обеспечения в области технологийи инструментов для управления профессиональнымикомандами программистов.
Received 10.01.19310
Technologies of Intelligence MultiagentInformation Proceccing with Blockchain for
ManagementVishniakou U.A., Shaya B. H., Al-Masri, A. H., Al-Haji S. K.
Belarusian State University of Informatics and RadioelectronicsMinsk, Belarus
Abstract—The tendencies of using multi-agent intelligent tech-nologies for information processing are given. The main ideasof building a distributed multi-agent system with distributedknowledge and distributed processing are shown. The structure ofmulti-agent system for monitoring sound information is done. Thedirections of using multi-agent intelligent systems in managementactivities using cloud and block chain technologies are presented.As the concept proposed the creation of a distributed tool platformbased on multiagent approach, combining semantic and blockchaintechnology.
Keywords—multi-agent technologies, distributed knowledgebases, distributed decision-making, cloud environment, tool plat-form
I. INTRODUCTION
Under multi-agent technology is understood the technologyof development and use of multi-agent systems (MAS) andmulti-agent management (MAM). The problems of controland distributed interaction in networks of dynamic systemsattract the attention of a large number of researchers. This isdue to the widespread use of multi-agent systems in differentareas, including automatic adjustment of parameters of neuralrecognition networks, transport management, distributed sensornetworks, control in communication networks, interaction ofUAV groups, management of mobile robots, protection ofinformation resources, etc. Distributed MA systems are usedthat perform actions in parallel, for which the task is dividingon parts between several computational threads. Such problemsarise not only in computer networks, but also in production net-works, service networks, transport and logistics networks. Withnatural constraints on communication, decentralized strategiesare able to effectively solve this type of problem [1-3]. Blockchain technology is another application layer working on top ofthe Internet Protocol stack [4]. It is proposed to use distributedblock chain tools to control the operation of intelligent MAS.
II. THE BASIS OF MAS
Multi-agent systems originated at the intersectionof system theory and distributed artificial intelligence.Open, active, developing systems are discussed, in whichattention is paid to the processes of interaction of agentsfor building systems with new qualities. MAS are builtas a union of individual intelligent systems based onknowledge [3]. The MAS consists of the following com-ponents: a set of agents working with objects; variety oftasks; a space in which there are agents and objects; theset of relations between agents; many agent actions (op-erations on objects). Agent management system (AMS)
is also an agent that controls access and use of the agentplatform [1].
The basis of the organization form of interactionbetween agents characterized by the combination of theirefforts to achieve the goal in the division between theirfunctions, roles and responsibilities is cooperation (C).This can be determined:
K = cooperation+ coordination +communication Un-der the coordination means the management of theassociations between actions. Communication betweenagents depends on the chosen protocol, which is a set ofrules that determine how to synthesize meaningful andcorrect messages.
In the MAC architecture, the main part is the domain-independent core, which includes such components: di-rect access service (provides direct access to the at-tributes of agents); message service is responsible forthe transmission of messages between agents and ker-nel systems; agent class library (part of the database)contains the classification of agents in the MAS; agentscommunity, where agents are located (this block providesfunctions for loading/writing agents and their propertiesand optimizes the work of agents with resources); on-tology is a subject knowledge base containing specificknowledge about objects and environment of functioning,represented in the form of a corresponding semanticnetwork [1, 2].
III. THE AGENT STRUCTURE AND THEIR USE
The basis of agent structure is the context, or serverenvironment, in which it is executed. Each agent has afixed identifier-name. In a server environment, you canrun not only the source agent, but also a copy of it.Agents are able to create their own copies, sending themto different servers for execution. When the agent arriveson the next server, its code and data are transferred tothe new context and erased at the previous location. Inthe new context the agent can do anything that is notprohibited there. Upon completion of the work in thecontext the agent may send itself in a different contextor upload sender address. Agents can also shut down
311
themselves or at the command of the server, which thenmoves them from the context to the storage location.
The structure of a typical agent includes inputs (in-ternal parameters of the agent and data on the stateof the environment), outputs (parameters affecting theenvironment and informing the user about the state ofthe environment and decisions made), the solver – thedecision-making procedure. The solver can be a fairlysimple algorithm or an element of an artificial intelli-gence system [2].
Multi-agent systems are used for the development ofinformation and industrial systems. In industry the MASare used to the solution of management automation ofcomplex systems, for the collection and processing ofinformation in games. Multi-agent technologies are ap-plicable in the management of mobile resources, as wellas in such areas as object design, industrial production[1].
In sources [5, 6] MAS deals with the applicationof automate the construction of intelligent knowledgebases and problem solvers. The resulting model of hybridknowledge bases, which ensures the compatibility ofpresent knowledge and can be represented in knowl-edge bases multi-level meta knowledge, to structure theknowledge base according to various criteria and toapply components of knowledge bases again [5]. Theagent-oriented model of the hybrid solver allows tobuild variety of MAS: for produc-tion, customer service,construction design [6].
IV. DESIGN OF MAS
The general methodology of the ascending evolution-ary design of MAC can be represented by a chain: <envi-ronment - functions OF Mac – role of agents – relationsbetween agents – basic structures of MAC-modification>.It includes the stages of: formulation of purpose (objec-tives of development) MAC; the identification of coreand support functions of agents; clarify the compositionof agents and the distribution of tasks among agents, thechoice of the architecture of the agents; the provisionof basic relationships between agents; determination ofpossible actions (operations) agents; analysis of real-life,real or anticipated changes in the environment. Whendesigning, the organization of agents can be consideredas a set of roles that are in a certain relationship witheach other and interact with each other [1].
MAS bottom-up design methodology requires a pre-liminary task of the initial functions, determining therange of their obligations to each other, the formation ofthe initial structures and its developing on the basis ofthe allocated functions and the study of the adequacy ofthese structures to the nature of the tasks in the selectedproblem areas.
The technique of top-down design is to determinethe social characteristics of MAS on a set of criteria,
the construction of the basic types of their organiza-tions, followed by the definition of requirements forthe architecture of agents. For artificial social systemsand communities, a top-down approach to organizationaldesign is put forward [2].
Agents can be integrated into cloud computing (CC)structures that contain specific functions for problemsolving, data processing, and management. They sup-port a natural mix of knowledge-based information andtechnology and can support the process of logical rea-soning (for example, including business regulations).They enable learning and self-improvement at both theinfrastructure level (adaptive routing) and the applicationlevel (adaptive user interfaces) [7, 8].
V. MAS STANDARDS AND PLATFORMS
There are several international approaches to creatinga MAS, the most famous of them are [1]: MASIF (ObjectManagement Group), which is based on the concept of amobile agent; FIPA (Foundations for Intelligent PhysicalAgents) specifications based on the intelligence of theagent, as well as standards developed by the researchsubsection Defense Advanced Research Projects Agency(DARPA), in particular Control of Agent Based Systems.Regarding mobility and intelligence of agents, mostexperts agree that mobility is the Central characteristic ofthe agent, intelligence is desirable, but not always strictlyrequired. [1, 2].
FIPA’s activities include joint research and develop-ment by its members of international specifications thatwill maximize the interaction between agent applications,services and equipment. FIPA specifications focus onenabling intelligent agent communication through stan-dardized agent communication and content languages.Along with the General basics of communication, FIPAalso specializes in ontology and negotiation protocols tosupport interaction in specific application areas (transportsupport, production, multimedia, networking) [1].
The OMG MASIF standard creates conditions forthe migration of mobile agents between MAS via stan-dardized CORBA IDL interfaces. DARPA initiated thework on the distribution of Knowledge Sharing Effort,as a result of which the agent programming languageswere divided into syntax, semantics and pragmatics:language KIF (Knowledge Interchange Format) – syntax;Ontolingua – language for defining shared ontologies (se-mantics); KQML (Knowledge Query and ManipulationLanguage) – a high-level interaction language (pragmat-ics). When you create a MAS is also used language ofcommunication between agents – Agent CommunicationLanguage (ACL) that specifies the types of messagesagents, the content and the ontology. Cooperation be-tween agents is achieved through a set of basic conceptsused in communications. The ontology is used as theApplication Programming Interface and defines the agentinterface.
312
At the technical level, communication between agentsis carried out by message transmission using the ap-plication layer Protocol (SMTP, TCP/IP, HTTP, IIOP).Alternatives to using ACL are a number of languagessuch as database languages (SQL), Distributed objectsystems (CORBA), Service languages, and Web lan-guages (XML, RDF, DAML).
The evolution of agent creation technologies requires:the development of semantics of communication lan-guages of ACL agents, the development of ontology;improving the use of metadata; declarative protocols (lan-guages for the definition of high-level protocols based onmore primitive); practical knowledge exchange betweenagents (mechanisms for the exchange of information andknowledge); the development of schemes and methodsfor controlling agent sys-tems (artificial markets, naturalselection, etc.) [1].
Agent platforms are one of the ways to build dis-tributed systems and allow user to describe and provideaccess to all applications running on the agent platformto the services they need. The functions of the agentplatform include the distribution of agents, audit of theirfunctioning and management.
MAS development is based on the following tools[2]: JADE (Java Agent Development Framework) - soft-ware environment for creating multi-agent systems andapplications that supports FIPA standards for agents. Itincludes the agent runtime environment, a library ofclasses that are used to develop agent systems, a setof graphical utilities for administration and monitoringthe life of agents, connected to the project in the Javalanguage. JADE agents can be different: from simple,reacting, to complex - mental. JACK Intelligent Agents isused as a Java platform for creating multi-agent systems.Just like the JADE platform, it extends Java with itsclasses. JACK is one of the platforms where the modelof agents ’logic based on beliefs-desires-intentions (Be-lief–desire–intention software model - BDI) and built-informal logical means of agents’ work planning are used.
The functionality implemented within the frameworkof the paradigm block chain can look like an integratedphysical level of calculations with many devices, on topof which there is a layer for servicing payments. Butit’s not just about payments, but about micropayments,a decentralized exchange, earning and spending tokens,getting and transferring digital assets, and drawing upand executing clever contracts - that is a full-fledgedeconomic layer that has not yet been available in theInternet [4].
VI. MAS FOR INFORMATION PROCESSING
The components of a multi-agent system of informa-tion defense are intelligent programs (protection agents)that implement the specified functions to provide therequired security class. They allow you to implement a
comprehensive security system used network software,operating systems and applications, increasing the secu-rity of the system to the required level. Within the frame-work of this research direction, architectures, models andsoftware prototypes of several MAS were developed:attack modeling, intrusion detection, intrusion detectiontraining, etc. [9]. The process of creating multi-agentsystems for any subject area, including the protection ofinformation in computer networks, involves the solutionof two tasks: (1) the creation of a «system core» of MAS;(2) cloning of software agents and the separation of thegenerated multi-agent system from the «system core» [1,9].
The architecture of the MAS intrusion detection(MAID) multiple instances of agents of different types,specialized for solving a subtask of intrusion detection.Agents are distributed among hosts of the protectednetwork, divided by types of tasks and exchange infor-mation for making coordinated decisions [9]. The eventagent (AE) preprocesses incoming messages to the host,captures important events to protect the information andforwards the selected messages to the appropriate specialagents. The identity and authentication agent (AIA) isresponsible for identifying the message sources andauthenticating them. The access control agent (ACA) reg-ulates users ’ access to network resources in accordancewith their rights and privacy labels of security objects.The agents AIA and ACA detect unauthorized access toinformation resources of the host interrupt connections,and processes events that are identified as unauthorizedand send messages to the agents to intrusion detection.The agents A-P1 and A-P2 (A-Patterns) are responsiblefor detecting individual «suspicious» events or obviousfacts of intrusion and making decisions regarding thereaction to these events (facts). Intelligent intrusion de-tection agents IA1 and IA2 implement a higher level ofprocessing and generalization of the detected facts. Theymake decisions based on reports of detected suspiciousbehavior and explicit attacks, both from their host agentsand from agents of other hosts [9].
VII. MAS FOR SOUND PROCESSING
The multi-agent system for monitoring sound infor-mation (MAMS) in the environment are a set of agentsfor sound transformation, agent for analysis of informa-tion received from them and agent for decision-making.MAMS implements the functions to ensure the requiredclass of protection of people (working or living) andallows to implement an environmental safety system.MAMS can handle noise levels in the urban space andhelp in learning noise pollution of various areas: insidethe building, in a public park or around the entire area,increasing the protection of the space to the required level[7].
MAMS implements the functions to ensure the re-quired class of protection of people (working or liv-
313
ing) and allows implementing an environmental safetysystem. The classification agent in MAMS can handlenoise levels in the urban space and help in learning noisepollution of various areas: inside the building, in a publicpark or around the entire area, increasing the protectionof the space to the required level. A conceptual schemawill be automatically enhanced in the transformationagent in order to help in conception and decision making.
VIII. CONCEPT OF IMAS DEVELOPMENT
The paper [10] analyzes the main developments in thefield of intelligent control and the main trends in thedevelopment of IMAS. As a result, a list of criteria andtheir values have been defined, which must be met bythe is support IA:
• multi-level monitoring of the environment, collecting in-formation about the state of the network from varioussources at different levels of monitoring: the level of thenetwork, servers and subsystems;
• adaptability, ability to detect modified implementations ofknown and new network attacks;
• proactivity, the presence of built-in mechanisms of reac-tion to the emergence of attacks;
• openness, the ability to add new analyzed resources ofthe information system;
• type of control. IP map should combine both centralizedand distributed management;
• security. IP map should have the means to protect itscomponents.
As a result, the following solutions for intelligent MASinformation processing (IMAC IP) are presented:
• structure and composition of IMAC IP includes agentsof workstations, servers, routers and networks and allowsto draw the conclusion about the state and prospects ofprotection development;
• the method of making a joint decision by agents, allowingto form the round table of agents, and on the basis oftheir result analysis of information obtained from varioussources, to assess the state of protection as a whole;
• a technique for detecting attacks using multi-agent tech-nologies that allows user to train a multi-agent system todetect attacks and use it to further detect new threats;
• evaluation of the effectiveness of all the proposed methodsusing the developed software solutions of the intelligentMA platform.
As trends and development concepts for the use ofintelligent and block chain technologies in the MASmanagement (MASM) is proposed:
• improving the architecture of the MASM in cloud envi-ronments, providing effective management in the condi-tions of uncertainty of the information environment;
• development of new management models in CC with IAbased on the choice of the optimal response to environ-mental events;
• improvement of instrumental program complexes forMASM with intellectual support of decision-making andresearch of efficiency of methods, models and algorithms;
• development of MAS technology of security managementfor intrusion detection, countering threats of violationof information security, assess the level of security ofinformation in CIS;
• development of theoretical foundations, models and toolsof cloud platform for designing intelligent systems ofMASM on the basis of semantic technologies;
• development of application software of workstations orsites for managers and marketers using block chain tech-nologies.
IX. CONCLUSION
1. The first direction of intelligent MAC IP development isthe further development of models, methods, architectures andsoftware to solve the problem of adaptation in the environment.
2. The second direction is the development of models, meth-ods, architectures and software for the collection, structuring ofinformation from the environment, the formation of specializedknowledge bases and decision support agent.
3. The third direction is the creation of a cloud-based toolplatform for the design of intelligent MACM based on semanticand block chain technologies.
REFERENCES
[1] Leyton-Brown K, Multiagent Systems: Algorithmic, Game-Theoretic andLogical Foundations. London: Cambridge University Press. 2009. 513 p.
[2] G., Rzevski Modelling large complex systems using multi-agent technol-ogy. Proc. of 13th ACIS International Conference on Software Engineering,Artificial Intelligence, Networking, and Parallel/Distributed Computing,August 8-10, Kyoto, Japan, 2012, pp. 434-437.
[3] Tarasov V. B. Agents, multi-agent systems, virtual communities: strategicdirection in the field of information science and artificial intelligence. Newsof artificial intelligence, 1998, No. 2, pp. 5-63.
[4] Swan M. Block chain. Scheme new economy "Olymp-Business", 2015.142 p.
[5] Davydenko, I. T. Models, methods and means of development of generatedknowledge bases on the basis of semantic compatibility of reusable com-ponents. Extended abstract of PhD dissertation.Belarusian State Universityof Informatics and Radioelectronics, Minsk, 2018.
[6] Shunkevich, D. V. Agent-oriented problem solvers of intelligent systemscomponent. Extended abstract of PhD dissertation.Belarusian State Uni-versity of Informatics and Radioelectronics, Minsk, 2018.
[7] Vishnyakou, U. A. Information management. Manual with the stamp ofEM Minsk: Bestprint, 2015. 305 pp.
[8] Fingar, P. Cloud computing – the business platform of the XXI centuryMoscow: Aquamarine Book, 2011. 256 p.
[9] Kotenko I. V. Technologies of computer security / I. V. Kotenko, R. M.Yusupov. Bulletin of the Russian Academy of Sciences, 2007, volume 77,No. 4, pp. 323-333.
[10] Vishnyakou, U. A. Development of intelligent control by using cloudtechnologies. Informatics, 2016, No. 2, pp. 113-120.
ИСПОЛЬЗОВАНИЕ ТЕХНОЛОГИИИНТЕЛЛЕКТУАЛЬНОЙМНОГОАГЕНТНОЙ
ОБРАБОТКИ ИНФОРМАЦИИ С БЛОКЧЕЙН ДЛЯСИСТЕМ УПРАВЛЕНИЯВишняков В.А., Шайя Б.Х.,
Эль Масри А.Х., Эль Хаджи С.К.
Приведены тенденции использования многоаген-тных ин-теллектуальных технологий для обработки информации.Показаны основные идеи построения распределенноймного-агентной системы с распре-деленными знаниями и распреде-ленной обработкой. Дана структура многоагентной системыдля обработки звуковой информации. Представлены направ-ления использования много-агентных интеллектуальных си-стем в управлен-ческой деятельности с использованиемоблачных и блокчейн технологий. В качестве концепциипредложено создание инструментальной распреде-леннойплатформы на базе многоагентного подхода, объединяющейсемантические и блокчейн технологии.
Ключевые слова: многоагентные технологии, распреде-ленные базы знаний, распределенное принятие решений,облачая среда, инструментальная платформа
Received 06.01.19
314
Efficiency of Intellectual System of SecureAccess in a Phased Application of Means ofProtection Considering the Intersection of the
Sets of Threat DetectionVladimir S. Kolomoitcev,
Vladimir A. Bogatyrev, Vladimir I. PolyakovITMO University
Saint-Petersburg, Russiadekskornis, [email protected], [email protected]
Abstract—We have conducted time delays estimationinduced by system of secure access in a stage-by-stage usesmeans of protection of information to detect and eliminatethreats for intellectual information protection system. Wasconsidered a different sequence of the stage-by-stage usesmeans of protection of information and intersection ofthe sets of threat detection and elimination. For systemof secure access is shown that the best option of systemprotection elements consistent use is the option in whichthe system elements are used sequentially from the least“simple” (which having a smaller area prevent securitythreats) to the “complex” (which having a large activityarea). At the same time, at low arrival rate, consistent useof connecting “complex” means to “simple” means givesclose results to the best options. However, the differencebetween them begins to grow rapidly with the increase inthe arrival rate, approaching the worst options – options inwhich “complex” means are at the center of the informationsecurity system. It shows a comprehensive estimating of theeffectiveness of the system of secure access in terms of theintroduced delays and information security.
Keywords—information protection, information security,computer system, system of secure access, system secure as-sessment, intellectual protection system, information threat
I. INTRODUCTION
Design of computer systems (CS) that can activelywithstand to information security (IS) threats using in-tellectual system of secure access (SSA) is one of thekey tasks of corporate system design [1], [2], [3], [4],[5].
An intellectual SSA have limitations to threat detectionespecially when they function in real time. Their capa-bilities are limited by the computing power of the meansof protection of information (MPI) and allowable delaysin detecting and eliminating IS threats (which largelydepend on the configuration MPI‘s for the SSA and thesequence of their application.
Information security threats can be of a differentnature, ranging from unauthorized access to a CS and
ending with getting infected with virus data on individualnodes of a CS, in order to create conditions for theinability of the CS to work properly. An MPI‘s toprevent threats to IS can be firewalls of various types,anti-virus means located at different levels of the CS,means of protecting against unauthorized access, andother MPI‘s. Each of these means, both individually andin the complex, is able to provide one or another level ofinformation security of the computer system as a wholeand its individual nodes with various delays in processingrequests received by the system.
The purpose of this work is to increase the effective-ness of the SSA based on the selection of variants of itsstructure on the stage-by-stage use of different MPI andlikely intersection of the sets of threat detection of theseMPI‘s.
The efficiency of designing SSA is defined in terms ofproviding them protection from threats of IS and delaysprocessing of incoming requests to IS.
The calculation of delays is especially important forintelligent protection systems that require complex infor-mation processing when detecting fuzzy decision-makingmodels under uncertainty [6], [7].
II. THE OBJECT OF STUDY
The object of the study discusses the SSA "Direct con-nection" involving connecting several elements (means orways) protection implemented in the form of hardware,software platforms or their combinations as part of aunified system of information protection [8], [9], [10].Typical SSA "Direct connection" is shown in “Fig. 1”.
The SSA "Direct connection" includes three key ele-ments: terminal nodes of the CS, MPI that are outsideof terminal nodes of the system and communicationchannel.
The objective of the SSA "Direct connection" is toensure IS of terminal nodes of the CS. For this in its
315
Figure 1. The system of secure access – "Direct connection".
composition used specialized hardware and software MPIaimed at solving problems of IS. MPI may also belocated on the terminal nodes of the CS. First, however,the process of IS within the CS rests on external remediesto improve performance of SSA and reduce processingload on end nodes of CS.
A communication channel carries out switching of theCS from the external network or to other areas of thecorporate network which uses other methods to ensureIS.
We consider a system of information protection im-plemented in the compute node with a certain set ofsoftware MPI. MPI that are used in the SSA are activatedstep-by-step (sequentially), in this regard, consider theprotection system as a queueing system with a step-by-step executing queries [11], [12]. Service process ofrequest in the system of information protection, includingthe R-stages, is shown in “Fig. 2”.
A request immediately leaves the system with prob-ability Pi (the MPI found and removed the threat of
Figure 2. Service process of request in the system of informationprotection, including the R-stages: V1, . . . , VR – the processing timeon the stages of the system of information protection; P1, . . . , PR−1 –the probability of passing the request to the i− th stage of the systemof information protection, i = 1,. . . , R.
IS) or arrives at the next (i + 1)-th stage of servicewith probability (1 − ˘Pi), after the completion of thei-th stage. The request leaves the system and beginsphased implementation of the next request from thequeue after service completion at stage R. We assumethat the service time of these stages has an exponentialdistribution.
III. ESTIMATION OF AVERAGE RESIDENCE TIME OFTHE REQUEST IN THE SYSTEM
Queuing system with stage-by-stage service “Fig. 2”is a special case of a queuing system of type M/G/1[12], [13], [14], [15], [16], [17]. The average residencetime of the request in the system T can be defined bythe Pollaczek–Khinchine equation, as shown in “(1)”:
T = x+ ρx1 + C2
v
2(1− ρ)(1)
where x — the average time of service request; ρ = λx— the coefficient of exploitation (ρ < 1 ), here λ— arrival rate; C2
v = σ2v/(x)2 — the square of the
coefficient of variation, and σ2v -– the dispersion of the
service time of the request.One way to improve the reliability and performance
of the SSA is the integration of MPI in clusters with adistribution of requests between nodes [19], [20], [21].Then, the arrival rate of requests that arrive in each ofM-systems will be divisible by M if the SSA includesM-systems, which serve incoming requests. In the result,the equation to calculate the average residence time ofthe request in the system shown in “(2)”:
T = x+ ρx1 + σ2
v/(x)2
2(M − ρ). (2)
We assume that the service time of these stages has anexponential distribution. Then, using the distribution ofCox (in accordance with “Fig. 2”), we obtain the Laplacetransform for the density of probability distribution ofservice time in the form “(3)” [13], [14]:
B(s) =µ1
s+ µ1P1 + (
R−1∑
i=2
Pi(
i−1∏
j=1
(1− Pj))·
·i−1∏
j=1
(µj
s+ µj) +
R∏
i=1
(µi
s+ µi)
R−1∏
j=1
(1− Pj)(3)
316
where µi – the intensity of service; Pi – the probabilityof removing the threat of IS on the i − th stage of theservice; R – the number of stages in SSA.
We can obtain the mathematical expectation and dis-persion of the average service time for the system,including the R-stages of service after the definitionof the Laplace transform for the density of probabilitydistribution of time. We will use the following equationto calculate the n − th initial moment of the randomvariable [13]:
Xn = (−1)nA∗(n)(0) (4)
The first derivative of B(s) corresponds to the firstinitial moment, and the mathematical expectation:
x =dB(s)
ds|s = 0 (5)
The second derivative of B(s) corresponds to thesecond initial moment:
x(2) =d2B(s)
ds2|s = 0 (6)
Given the Laplace transform, receiving the first (x)and second (x(2)) initial moment, it is possible to findthe dispersion:
σ2v = x(2) − x2 (7)
For example the calculation, suppose a system of in-formation protection of CS includes three MPI (R = 3).Thus, the average time R-stage service defined as:
B‘(s) =P1µ1
(s+ µ1)2+
µ1µ2P2(1− P1)
(s+ µ1)(s+ µ2)2+
+µ1µ2µ3(1− P2)(1− P1)
(s+ µ1)(s+ µ2)(s+ µ3)2+
+µ1µ2P2(1− P1)
(s+ µ1)2(s+ µ2)+
µ3
s+ µ3×
× (µ1µ2(1− P1)(1− P2)
(s+ µ1)(s+ µ2)2+
+µ1µ2(1− P1)(1− P2)
(s+ µ1)2(s+ µ2))
(8)
Knowing that µi = V −1i and having s = 0, we get the
average service time, as shown in “(9)”:
x = V1P1 + (V1 + V2)(1− P1)P2+
+ (V1 + V2 + V3)(1− P1)(1− P2)(9)
where Vi – service time i− th MPI.
The second initial moment for SSA that includes 3-step defined as:
B“(s) =µ1
(s+ µ1)(µ2µ3(1− P1)(1− P2)
(s+ µ2)2(s+ µ3)2)+
+µ1µ2( 1
µ2+s+ 1
µ3+s)(1− P1)(1− P2)
(s+ µ2)(s+ µ3)2+
+2µ2(P2)(1− P1)
(s+ µ1)2(s+ µ2)+
+2µ2µ3(1− P1)(1− P2)
(s+ µ2)(s+ µ3)×
× (1
(s+ µ1)2+
1
(s+ µ2)2+
+1
(s+ µ1)(s+ µ2))+
+2µ1(1− P1)P2
(s+ µ2)3+
2µ1(1− P1)P2
(s+ µ1)(s+ µ2)2+
+2P1
(s+ µ1)2+
2µ2µ3(1− P1)(1− P2)
(s+ µ2)(s+ µ3)3+
+µ2µ3(1− P1)(1− P2)
(s+ µ1)(s+ µ2)(s+ µ3)
(10)
Knowing that µi = V −1i and having s = 0, we get the
dispersion of the service time, as shown in “(11)”:
σ2v = 2(V1(P1 + (1− P1)(P2)) + V 2
2 P2(1− P1)
+ (V 21 + V1V2 + V 2
2 )(1− P1)(1− P2)+
+ V 23 (1− P1)(1− P2)) + V1V2P2(1− P1)+
+ 2V3(V1 + V2)(1− P1)(1− P2)− x2(11)
IV. DETERMINING THE PROBABILITY OF DETECTINGTHREATS IN A STAGE-BY-STAGE USES MEANS OF
PROTECTION OF INFORMATION
The probability of detection of threat after applying rMPI‘s can be defined as:
Pr = 1−r∏
i=1
(1− pi) (12)
There are some sets:• H -– the set of threat of IS which need to be
addressed in the context of a specific CS;• E – the set of threat of IS which is capable of
detecting (and with some probability, eliminate) theset of R means (or ways) used in the system ofinformation protection;
• Ai – the set of threat of IS which is capable ofdetecting and eliminate i− th MPI of the system ofinformation protection.
Define:• Li = |Ai|/|E| -– the proportion of threats of total
set of threats of IS detected and eliminated by thei− th element;
• li...m = |Ai ∩ Aj ∩ ... ∩ Am|/|E| – the proportionof threats of total set of threats of IS detected and
317
eliminated elements i, j to m (using different fromeach other methods and/or algorithms) used in thesystem consisting of several elements;
• I = |E|/|H| the coefficient of "coverage" of threatsof IS (which can be susceptible to the CS) detectedby the elements used in the system (E ⊆ H);
• |E|, |H|, |Ai| a cardinal number (number of threats)of sets, respectively E, H , Ai.
We assume that all threats are not equivalent amongthemselves, that is, the losses that they can cause the CSare different.
In the system highlight areas to detect and eliminatethreats of IS: an area where work can be conducted onlyone element that is part of the system of informationprotection of CS and the area where you can work severalelements of the system of information protection.
In general the intersection of the sets of threats of ISto the system of information protection, which includesthree elements (with the proportion of the threats ad-dressed by each element or group of elements) is shownby Venn diagram in “Fig. 3”.
Figure 3. The proportion of the threats eliminate by each element orgroup of elements included in the system.
Given that the system of information protection usesMPI have the intersection of the sets of detected threatsof IS, then get that the probability of eliminating thethreat Pi for the i− th service stage “(13)”:
Pi = I ·W · (lipi +
j<i∑
i=1
(lji(pipj) +
q<i∑
q=1
lqji(pipjpq)+
+ ...+
m<i∑
m=1
lmt...i(pipj ...pm))...)
(13)
where W = λT /λ – the proportion of threats of IS in thechannel of communication, here λT and λ – arrival rateof threats of IS and total arrival rate (including threats ofIS), accordingly; li – the proportion threats of total set of
threats of IS detected and eliminated by the i−th elementof the system of information protection that consists ofR-elements; li. . .m – the proportion of threats of totalset of threats of IS detected and eliminated elements i, jto m used in the system consisting of several elements;pi – the probability a threat is detected by the i−th MPI;pj – the probability defined as pj = 1 − pj ; i, j, . . . , t– the ordinal numbers of the elements of the system ofinformation protection. Thus, using “(13)” we get P1 andP2, as shown in “(14)” and “(15)”:
P1 = I ·W · l1p1 (14)
P2 = I ·W · (p1(L2 − l12) + l12p1p2)) (15)
Here L1 = |A1|/|E| and l12 = |A1 ∩A2|/|E|.V. COMPREHENSIVE ASSESSMENT OF THE
EFFECTIVENESS OF THE SECURE ACCESS SYSTEM
We can use a comprehensive indicator of effectivenessexpressing the normalized average time savings beforedetecting and eliminating a threat relative to the max-imum allowable delay time introduced by the SSA fora comprehensive assessment of the effectiveness of theSSA "Direct Connection". The comprehensive indicatorof effectiveness is shown in “(16)”:
QS =To − TT0
· PS (16)
Here T0 - maximum allowable time of the request inthe SSA; T - average time the request in the SSA; PS- information security of the system. The informationsecurity of the system can be found using the equation forestimating the probability of detecting a threat of infor-mation protection by the information protection systemconsisting of R-elements (equation extension “(13)” forthe case of estimating the probability of detecting a threatto the entire system, rather than individual stages of itsoperation) – “(17)” [22].
PS = I ·W ·R∑
i=1
((lipi +
j<i∑
i=1
(lji(1− pipj)+
+
q<i∑
q=1
lqji(1− pipjpq)+
+ ...+
m<i∑
m=1
lmt...i(1− pipj ...pm))...)
(17)
VI. EXAMPLE OF CALCULATION OF THE AVERAGETIME OF DETECTING THREATS IN DIFFERENTSEQUENCES OF APPLICATION OF MEANS OF
PROTECTION OF INFORMATION AND SYSTEMEFFECTIVENESS INDEX
Let the tenth part in the incoming to CS data ismalicious (threat to CS) and can be detected used inits elements of information protection (W = 0.1), and,also, we assume that I = 1 (all MPI‘s used as part of
318
a system of information protection cover all the existingthreats against the CS). The service time of elementsof the system of information protection m1,m2,m3 :V1 = 0.0025 c, V2 = 0.004 c, V3 = 0.0075 c. Theprobability of eliminating the threat of IS of each ofthe elements of the system of information protection:p1 = 0.9 , p1 = 0.95 , p1 = 0.925 . The proportion ofthreats of total set of threats of IS detected and eliminatedelements of system of information protection: L1 = 35%, L1 = 50% , l12 = 15%.
When completing the system with three means ofprotection, there are the following options for theirconsistent application:
• b1 : (m1,m2,m3);• b2 : (m1,m3,m2);• b3 : (m2,m1,m3);• b4 : (m2,m3,m1);• b5 : (m3,m1,m2);• b6 : (m3,m2,m1).Substituting in the equation “(2)”, of equations “(9)”,
“(11)”, “(14)” and “(15)”, we get, the average servicetime of request of the SSA – corresponding to the averagetime a threat is detected. The dependence of the averagetime of request in the system on the arrival rate fordifferent options of placing of elements of the systemof information protection shown in “Fig. 4”.
Figure 4. The dependence of the average time of request in the systemon the arrival rate. Option b1 and b5 – black and grey lines, accordingly;option b3 and b4 – grey and black the dash-dotted line, accordingly;option b2 and b6 – grey and black the dotted line, accordingly.
From “Fig. 4” we can see that option b3 sequence ofapplication of various MPI in the system of informationprotection the best. It involves a sequential arrangementof the elements of the system of information protectionfrom “simple” (with a smaller area of detection andelimination of threats of IS, but with greater speed)to a more “complex” (a larger area of detection andelimination of threats of IS, but slower). Option b1 has,
close to this embodiment the result of the placement ofthe elements of the system of information protection. Atthe same time, the difference between options b1 and b3increases with increase in the intensity of the arrival rate.A similar pattern is observed when comparing options b2and b4, and options b5 and b6. The options b2 and b4, thesequence of application of various MPI have significantlyworse performance of the average time of request in theSSA (threats detection). This is because in the center (inthis case, second) stage of work the system of informa-tion protection is the most “complex” MPI (thus, it isnecessary to work with almost partially unfiltered arrivalrate) in each of these accommodation options elements.At the same time, at low arrival rate, consistent use ofconnecting “complex” means to “simple” means (optionb5 and option b6) gives close results to the best options.However, the difference between them begins to growrapidly with the increase in the arrival rate, approachingthe worst options – options in which “complex” meansare at the center of the information security system(option b2 and option b4).
For the option of building a SSA "Direct connection"(b1) we get the following graph of the effectiveness ofusing SSA from 1 to 3 MPI‘s. “Fig. 5”.
Figure 5. The dependence of the system effectiveness index on thearrival rate. Three MPI‘s option – black lines; two MPI‘s option(m2,m1 and m3,m1) – grey and black the dotted line, accordingly;one MPI option (m2, m3) – grey and black the dash-dotted line,accordingly.
As can be seen from the graph (“Fig. 5”) with in-creasing arrival rate the efficiency of the SSA using athree MPI‘s becomes lower than the use of two MPI‘s.Similarly, with a further increase in the arrival rate, itwill be more efficient to use one MPI.
CONCLUSION
It was determined the average time of detecting threatsin different sequences of a stage-by-stage uses of various
319
means of protection information in the system of secureaccess. We raised the assumption that the service timeof each step is a random variable that has exponentialdistribution.
It is shown that a variant of a sequential arrangementof the elements of the system of information protectionfrom “simple” (with a smaller area of detection andelimination of threats of information security, but withgreater speed) to a more “complex” (a larger area of de-tection and elimination of threats of information security,but slower) is preferable in terms of providing the leastvalues for average time of threat detection.
The effectiveness of the system of secure access usinga different number of information security tools is shown.
REFERENCES
[1] Koren, I.Fault tolerant systems. Morgan Kaufmann publications,visit our San Francisco 2009 378 p
[2] Aysan H. Fault-tolerance strategies and probabilistic guaranteesfor real-time systems Mälardalen University, Västerås, Sweden.2012. 190 p
[3] Aliev T.I. The synthesis of service discipline in systems withlimits // Communications in Computer and Information Science.2016. V. 601. P. 151–156. doi: 10.1007/978-3-319-30843-2_16
[4] Sorin D. Fault Tolerant Computer Architecture. Morgan & Clay-pool 2009 . 103 p
[5] A.G., Fedosovsky M.E., Maltseva N.K., Baranova O.V., ZharinovI.O., Gurjanov A.V., Zharinov O.O. Use of Information Technolo-gies in Design and Production Activities of Instrument-MakingPlants//Indian Journal of Science and Technology, IET - 2016,Vol. 9, No. 44, pp. 104708
[6] Survivable Network Systems: An Emerging Discipline / R.J. Ellison et al. Software Engineering Institute. 1997 URL:http://www.cert.org .
[7] Kozachok A. V., Kochetkov E. V., Tatarinov A. M. CON-STRUCTION HEURISTIC MALWARE DETECTION MECH-ANISM BASED ON STATIC EXECUTABLE FILE ANALYSISPOSSIBILITY PROOF // Herald of computer and informationtechnologies - 2017. - No 3 (153). - pp. 50 – 56. DOI:10.14489/vkit.2017.03.pp.050-056
[8] Domarev V.V. The security of information technology. Systemsapproach.K: DiaSoft, 2004, 992 pp
[9] Schumacher M. Security Engeneering with Patterns, LNSC2754.Springer-Verlang Berlin Heidelberg, 2003, 87-96 pp
[10] Kolomoitcev V. S., Bogatyrev V. A. The Fault-Tolerant Structureof Multilevel Secure Access to the Resources of the Public Net-work // Communications in Computer and Information Science.2016. V. 678. P. 302 – 313
[11] L. Kleinrock. Communication Nets: Stochastic Message Flowand Design. — McGraw-Hill, 1964. — 220 p. — ISBN 978-0486611051.
[12] L. Kleinrock. Queueing Systems: Volume I – Theory. — NewYork: Wiley Interscience, 1975. — 417 p. — ISBN 978-0471491101.
[13] L. Kleinrock. Queueing Systems: Volume II – Computer Appli-cations. — New York: Wiley Interscience, 1976. — 576 p. —ISBN 978-0471491118.
[14] L. Kleinrock, Farok Kamoun. Hierarchical Routing for Large Net-works, Performance Evaluation and Optimization. — ComputerNetworks 1 (3): 155–174. — 1977.
[15] L. Kleinrock, Richard Gail. Queueing Systems: Problems andSolutions. Wiley-Interscience. — 1996. — 240 p. — ISBN 978-0471555681.
[16] Harchol-Balter, M. (2012). "Scheduling: Non-Preemptive, Size-Based Policies". Performance Modeling and Design of ComputerSystems. p. 499. doi:10.1017/CBO9781139226424.039. ISBN9781139226424.
[17] Harchol-Balter, M. (2012). "Scheduling: Preemptive, Size-BasedPolicies". Performance Modeling and Design of ComputerSystems. p. 508. doi:10.1017/CBO9781139226424.040. ISBN978113922642.
[18] Manuel, Laguna (2011). Business Process Modeling, Simu-lation and Design. Pearson Education India. p. 178. ISBN9788131761359. Retrieved 6 October 2017.
[19] V.A. Bogatyrev, S.A. Parshutina, “Efficiency of redundant multi-pathtransmission of requests through the network to destinationservers”,Communications in Computer and Information Science,vol. 678, 2016, pp. 290-301.
[20] V.A. Bogatyrev, S.A. Parshutina, N.A. Poptcova, A.V. Bo-gatyrev,“Efficiency of redundant service with destruction of ex-pired and irrelevant request copies in real-time clusters”, Commu-nications in Computer and Information Science, vol. 678, 2016,pp. 337-348.
[21] Bogatyrev V.A., Vinokurova M.S. Control and Safety of Op-eration of Duplicated Computer Systems//Communications inComputer and Information Science, IET - 2017, Vol. 700, pp.331-342.
[22] Kolomoitcev V.S., Bogatyrev V.A. A Fault-tolerant Two-tier Pat-tern Of Secure Access ’Connecting Node’ // ACSR-Advances inComputer Science Research - 2017, Vol. 72, pp. 271-274.
ЭФФЕКТИВНОСТЬ ИНТЕЛЕКТУАЛЬНОЙСИСТЕМЫ БЕЗОПАСНОГО ДОСТУПА ПРИПОСЛЕДОВАТЕЛЬНОМ ПРИМЕНЕНИИ
СРЕДСТВ ЗАЩИТЫ С УЧЕТОМПЕРЕСЕКАЕМОСТИМНОЖЕСТВ
ОБНАРУЖЕНИЯ УГРОЗ
Коломойцев В.С., Богатырев В.А., Поляков В.И.
Для интеллектуальных систем защиты информацииопределены временные задержки, вносимые системойбезопасного доступа с учетом различной последо-вательности поэтапного применения средств защи-ты и пересекаемости множества обнаруживаемых иустраняемых ими угроз. Для системы безопасногодоступа показано, что лучший вариант последователь-ного применения элементов системы защиты – тот,при котором элементы системы применяются после-довательно от наименее "слабых"(имеющих меньшуюобласть предотвращения угроз защиты информации) кнаиболее "сильным"(имеющих большую область рабо-ты). В то же время при низкой интенсивности входногопотока последовательное подключение от «сложных»средств к более «простыми» дает близкие результатык лучшим из исследованных вариантам построения.Однако разница между ними начинает быстро растис увеличением интенсивности входного потока, при-ближаясь к худшим вариантам - вариантам, в которых«сложные» средства находятся на центральных этапахработы системы защиты информации. Показана ком-плексная оценка эффективности системы безопасногодоступа по показателям вносимых задержек и инфор-мационной защищенности.
Received 29.12.18
320
Method of development of information securityexpert system
Tynarbay MarzhanL.N.Gumilyov Eurasian National University
Astana, [email protected]
Abstract—In the modern world, information technologyis used in almost all spheres of society, so informationsecurity is particularly relevant. There are various methodsof solving information security problems, one of which isthe use of expert systems. This article discusses the methodof development of information security expert system onExsys Corvid.
Keywords—information security, expert system
Credit card information leaks, identity theft, ran-somware, intellectual property theft, privacy breaches,denial of service-these information security incidentshave become common news. Among the victims are thelargest, wealthiest and most secure enterprises: govern-ment agencies, large retail chains, financial institutions,even manufacturers of information security solutions.Among the threats to the organization can be identified:
• Theft of confidential information, site deface, phish-ing, ransomware.
• Data loss due to natural phenomena or accidents.Security is closely connected with IT infrastructure
management: a well-managed network is more difficultto hack than a poorly managed one. To understand howwell an organization protects information, the followingquestions arise, such as do you know what do youremployees connect to their computers? What devicesare connected within the local network? Do you knowwhat software is used in your information systems?Did you configure your computers to meet informa-tion security requirements? Do you control employees’ access to confidential information or those who haveelevated access rights in the systems? Do your employeesunderstand their role in protecting your organization frominformation security threats? There are various methodsof solving IS problems, one of which is the use of expertsystems (ES). The possibility of using expert systems tosolve problems of information security has become ofinterest to specialists in information security due to thethe rapid development of information technology, hencethe emergence of new types of threats. Already expertsystems are used to solve some problems of informationsecurity:
• risk assessment and threat modelling;• antivirus software;• audit of information security of the enterprise.
Business disruptions in the form of data breaches,hacking, hijacking of website or social network accountsand IT infrastructure threats are part of the new businessreality for organizations in any sector. The use of EScontributes significantly to the detailed analysis and eval-uation of IS and the protection of the organization and itsinformation assets from current and future cyber threatsby specific information security specialists in variousorganizations without the involvement of additional andmore qualified personnel. The main purpose of ES is thatthey act as a kind of assistant or amplifier of intellectualactivity of a specialist in a particular subject area.
The generalized structure of the expert system isshown in Fig. 1. It should be noted that real ES canhave a more complex structure, but the blocks shown inthe figure are certainly present in any real expert system,since they represent the standard of the modern structureof the ES.
Figure 1. Structure of the expert system
Main components of IT used in the expert system:User interface, knowledge base (KB), knowledge engi-neer (ornithologist, interpreter engineer, analyst), expert,solver General scheme of interaction between the cre-ators of the expert system is shown in Fig. 2.
In the course of work on the creation of ES, acertain technology of their elaboration has developed,including the following six stages: identification, concep-tualization, formalization, implementation, testing, trial
321
Figure 2. Interaction of the creators of the expert system
operation (Fig. 3). At the identification stage, the tasksto be solved are determined, the development goals areidentified, experts and types of users are determined.At the stage of conceptualization, a meaningful analysisof the problem area is carried out, used concepts andtheir interrelations are revealed, the methods of problemsolving are determined.
Figure 3. The stages of development of expert systems
At the stage of formalization, the methods of rep-resentation of all types of knowledge are selected anddetermined, the basic concepts are formalized, the waysof interpretation of knowledge are determined, the workof the system is modeled, the adequacy of the goalsof the system of fixed concepts, methods of decisions,means of representation and manipulation of knowledgeis assessed.
At the stage of implementation, the expert fills theknowledge base. Due to the fact that the basis of ES isknowledge, this stage is the most important and the mosttime-consuming stage of ES development. The processof acquiring knowledge is divided into the extraction ofknowledge from the expert, the organization of knowl-edge that ensures the effective operation of the system,and the presentation of knowledge in the form of aunderstandable ES. The process of acquiring knowledgeis carried out by a knowledge engineer based on theanalysis of the expert’s activities to solve real problems.The method of development of expert systems is shownin Fig. 4.
However, already at the initial stages, serious funda-
Figure 4. Methods of development of expert systems
mental difficulties have emerged that prevent the widerspread of ES and seriously slow down and complicatetheir development. They are quite natural and followfrom the very principles of the development of ES(tab. I).
Table IPROBLEMS ENCOUNTERED IN THE DEVELOPMENT OF ES
Expert Error in expert knowledge such asincorrect and incomplete knowledge
Knowledge engineer
Semantic errors due to differentinterpretations of the meaning of theknowledge engineer and the expert;Incomplete knowledge of the expert.
Knowledge base
Syntax errors in the forms ofknowledge representation;Errors in content due to incorrect andincomplete knowledge, as well asuncertainty in rules and facts
Logical output machineErrors in the logical output machineand in other software tools of expertsystems
The output target
Inference errors due to incorrectprioritization of rules, interaction ofrules and errors in the knowledge base;Error due to non-monotonic inference.
A lot of tools are presented on the market for thedevelopment of ES. One of the leaders is a systemEXSYS CORVID.
Consider the example regarding the protection of in-formation of any organization. Hereinafter there will bea description on how to use the Corvid Exsys shell tocreate an ES.
In the beginning, in order to move forward on the issueof information security, it was necessary to deal withthe local network, connected devices, critical data andsoftware. Without a clear understanding of what you needto protect, it will be difficult for you to ensure that youare providing an acceptable level of information securi.
Key issues that are necessary for our system:Do you know what information needs to be protected?
Where is the most important information stored on yournetwork? Do you know which devices are connected toyour network? Do you know what software is installed onemployees ’ computers? Do your system administratorsand users use strong passwords? Do you know which
322
online resources are used by your employees (i.e., work-ing or sitting in social networks)? Consider the question:do you Know what information you need to protect?Where is the most important information stored on yournetwork?
To solve this problem, the variable "Know your infras-tructure"was set.
In the simplest case, you can write only 2 rules:
When all necessary variables are defined, logicalblocks are built (Fig. 5), which describe the knowledgein the system. A logical block can contain one or morelogical trees and/or rules.
Figure 5. Window logic blocks in the system Exsys Corvid
The user dialog after starting the ES will look as shownin Fig. 6 and Fig. 7.
The result is a very flexible and powerful developmentenvironment that can be quickly explored and imple-mented.Thus, a prototype was developed using ExsysCorvid, but the prototype should be improved with thehelp of new knowledge.
The expert system can be used to analyze and con-figure information security systems. This conclusion ismade on the basis of its main characteristics, proper-ties, analyses, methods of development. However, for
Figure 6. Window logic blocks in the system Exsys Corvid
Figure 7. Window logic blocks in the system Exsys Corvid
use in real-world tasks requires a fairly large data andknowledge base. Further work will be directed towardsfinding the sources of this data and knowledge. Also,it is possible to carry out additional studies to furtherdefine the task, by a more detailed description of theinput data. If you develop an expert system that willhave the following features:
• automation of risk assessment procedures;• the assessment should be based on the established
list of parameters;• low requirements to qualification of the expert;• presentation of the final assessment in a visual form;• ability to easily adapt to the requirements of new or
updated regulatory documents on is;• formation of a list of recommendations to improve
the organization’s is system based on the results ofthe program.
With the above features, the expert system will mosteffectively assess the risks of violation of is organiza-tions.
323
REFERENCES
[1] Sapozhnikov A. Yu., Krivosheev I. A. Application of ex-pert systems in the design process of aviation gas tur-bine engines // Molodoi uchenyi. 2009. 12. pp. 90-97. URLhttps://moluch.ru/archive/12/972/.
[2] D.I. Muromtsev. Exsys Corvid expert system shell. SPb: SPb GUITMO, p.69, 2006.
[3] Kirilov P., Information security guidelines for small and medium-sized businesses (SMB) - URL https://habr.com/post/348892/
[4] Giarratano George., Riley G. Expert systems. Principles of de-velopment and programming. M.: Izdat. house ”Williams”, 2007.1152 p.
[5] E. N. Sozinova Application of expert systems for analysis andevaluation of information security. Young scientist, 2011, 10,Vol.1. Pp. 64-66. URL https://moluch.ru/archive/33/3766/ (ac-cessed: 10.01.2019)
МЕТОД РАЗРАБОТКИ ЭКСПЕРТНОЙСИСТЕМЫ ИНФОРМАЦИОННОЙ
БЕЗОПАСНОСТИ
Маржан Тынарбай
В современном мире информационные технологиииспользуются практически во всех сферах жизни об-щества, поэтому информационная безопасность оченьактуальна. Существуют различные методы решенияпроблем информационной безопасности, одним изкоторых является использование экспертных систем.В данной статье рассматривается методика разработкиэкспертной системы информационной безопасностина Exsys Corvid.
Received 10.01.19
324
Choice of liver failure treatment usingset-theoretic models
Nikolay A. Blagosklonov and Boris A. KobrinskiiInstitute of Artificial Intelligence Problems
Federal Research Center "Computer Science and Control" of the Russian Academy of SciencesMoscow, Russia
[email protected], [email protected]
Abstract—The article presents the set-theoretic models of liverfailure, allowing the diagnosis of the disease forms and personalizedselection of traditional and new methods of treatment. Separatetypes are presented in the form of 5 sets which correspond tothe known clinical forms of the studied pathology: "chronic liverfailure", "acute-on-chronic liver failure", "acute liver failure","fulminant liver failure" and "drug-induced liver injury". Themodels allow to take into account the dynamics of transition fromone form to another in the process of developing the disease withcontinued influence of negative factors. The proposed approachallows the difference in the clinical manifestations of the pathology,the fuzzies of the transitional disease states, specific and non-specific signs of etiologically and pathogenetically different forms,which are the criteria for selecting similar cases to choice the mosteffective therapies. Personalized treatment is based on comparinga patient with a specific subset of the multiset.
Keywords—set-theoretic model, multiset, approximate sets,meta-analysis, liver failure, personalized therapy
I. INTRODUCTION
Liver failure is receiving increasing attention in studiesin different countries, despite the relatively rare morbid-ity due to extremely high patient mortality. Acute liverfailure is a life-threatening disease in which liver fails tofunction normally. The sudden loss of the synthetic anddetoxifying functions of the liver leads to jaundice, en-cephalopathy, coagulopathy, and multiorgan failure. Theetiology of liver failure is extremely variable. Mortalityreaches 40–50%. Primary care depends on timely recog-nition of the condition and early detection of etiology.Treatment includes intensive therapy, support for specificetiology treatment, if any, and early identification ofcandidates for liver transplantation. Liver transplantationremains the only therapeutic intervention with provenefficacy of survival in patients with irreversible liverfailure. Activities aimed at combating various types ofhepatitis and medicinal injuries of the liver will helpreduce the incidence and mortality from liver failure [1].
However, there are no universal methods for treatingthis pathology [2]. This is largely determined by therelevance of this study, involving the search for adequatemethods of personal therapy, based on taking into ac-count the specific characteristics of disease progressionin individuals.
There are numerous forms of the disease: acute, fulmi-nant, chronic, acute-on-chronic and a condition precedingliver failure – drug-induced liver injury. All of them
have similar symptoms, the main difference lies in thetiming of the development of hepatic encephalopathyregarding the first manifestations of jaundice, excludingdrug-induced liver injury [3]. It is possible to transferfrom one form of liver failure to another in the processof developing the disease and the continuing impact ofadverse factors. Thus, physicians deal with fuzzy mani-festations of the disease, which creates serious difficultiesfor the diagnostic plan.
Clinical trials of new approaches to the treatmentof this disease are constantly being conducted. Knowl-edge of modern methods of treatment by the attendingphysician can increase the survival of patients with liverfailure. The number of new publications is so huge thatknowing with them seems impossible for a practitioner.This problem can be solved by developing a system thatwill find similar cases in the literature and select themethod of treatment that most effectively showed itselfon a sample of similar patients.
To implement such a system, two models were devel-oped, combining different variants of the course of thedisease and methods of treatment. In the present study, aset-theoretic approach was chosen for building models,with the help of which various forms of the disease andrelations defined on them are described.
In the present formulation of the problem, syndromesand symptoms were used as elements of the sets ofsymbolic agents, as well as associated diseases, thepresence or absence of which in the patient is essen-tial for the formation of the clinical picture of liverfailure. The concept and description of sets from thestandpoint of different authors is quite diverse. The setsof signs presented in this study are a prerequisite foran accurate diagnosis and choice of treatment tactics.Since the model includes various forms of the disease,which have a similar set of manifestations, the diagnosticcriteria are repeated several times with minor changes.These differences must be considered in the differentialdiagnosis. Due to the fact that the elements will berepeated, the set-theoretic approach with the presentationof liver failure as a multiset is chosen as the basis of themodel.
325
A multiset is a set that allows the possibility ofmultiple presence of elements, that is, the existence ofseveral identical instances of the same element [4], [5].Due to the fact that hepatic insufficiency is characterizedby blurred clinical manifestations of individual forms andfeatures of a particular patient and different etiologicalcauses, the multi-set model includes fuzzy sets. Althoughany set can be represented as a continuous image ofspace in which all sets are closed. In this sense, alllocally connected continua are Jordan continua, that is,continuous images of a segment of a straight line [6]. Ina certain sense, one can also speak of the connectednessof the sets under consideration. According to WilliamWeiss [7], set theory is a true study of infinity.
II. CLINICAL AND FORMAL PROBLEMS OF FUZZINESSOF PATHOLOGICAL MANIFESTATIONS
In approximate sets, the boundary region allows tosimulate inaccuracy, and improving accuracy means re-ducing the boundary region. The theory of approximatesets [8] provides formal means for working with incom-plete or inaccurate information in terms of three-valuedlogic [9].
Medical information has a certain degree of fuzziness,determined by the possibility of an atypical course ofthe disease. The specific characteristics of individualscreate a situation of fuzziness in the manifestation ofdiseases. Based on the diverse knowledge of pathologicalmanifestations, a model of the disease is formed, whichincludes various forms and variants of development. Inother words, a model is a set that includes the interpre-tation of some symbols of relations and constants, whichmay or may not be present in specific cases. The courseof the disease can be characterized by the emergence ofnew signs, which does not exclude the transition of indi-vidual elements (objects that characterize the conditionof patients) from one subset to another.
In computer science, the characteristic function ofa fuzzy set is considered in the range from 0 to 1[10]. Thus, using the concept of fuzziness in medicalknowledge, it becomes possible to ensure that subsequentfeatures take into account the clinical characteristics ofindividual patients.
Consider also the concept of granulation as the abilityto operate with data and knowledge at various levels ofdetail as interval mathematics. It should be rememberedabout the general principle of granulation L. Zadeh [11]:to work effectively with inaccurate information shouldchoose the largest granules in accordance with the per-missible level of inaccuracy. This will help to reflect thelevel of generalization-detail when considering specificprocesses. One of the main components of the theory ofgranulation [12] are formal models of granules. These aresubsets, clusters, neighborhoods, multisets, approximatesets, fuzzy sets, etc.
The process of designing granules can be downwardor upward. In the descending process, a universal set istaken as a basis, which is divided into a family of subsets.In the upstream process, the original subset of objects isgrouped into a granule, and then the smaller granulesare combined into larger ones. This is similar to diseasesubclasses and classes. An important characteristic offuzzy logic is that any theory can be fuzzified and,therefore, generalized by replacing the notion of a clearset with the notion of a fuzzy set. The win from thefuzzification is the greater generality and the best fit ofthe model to reality [13]. In the same article, attentionwas drawn to the fact that fuzzy logic underlies themethods of working with inaccuracy, granular structure(granulation) of information, approximate reasoning, andcomputing with words.
III. SET-THEORETIC MODELS OF PATIENT ANDTREATMENT OF LIVER FAILURE
A formal presentation of the problem of liver fail-ure treatment, including various clinical forms, and ap-proaches to the choice of appropriate treatment tactics,it seems expedient to consider from the standpoint of settheory.
Accordingly, in the present study, two mathematicalmodels were developed: a diagnostic "Model of a patientwith liver failure" and a model for selecting optionsfor targeted therapy, called the "Liver failure treatmentmodel". Both models are based on the concept of amultiset and include fuzzy sets and subsets, which aredifferent forms of liver failure. Their fuzzies determinesthe difficulty of deciding on the choice of the mosteffective therapy.
As a result, 5 sets were formed that characterizevarious clinical forms of liver failure: "chronic liverfailure", "acute-on-chronic liver failure", "acute liverfailure", "fulminant liver failure" and "drug-induced liverinjury". When establishing relations between the fiveinitially distinguished sets, it was considered expedient togroup them into two contiguous, though not intersectingsets: “Form of chronic liver failure” and "Form of acuteliver failure". Inside each of the newly formed unitedsets, there are subsets that are closest in composition tothe elements, which were initially considered as separatesets. This is explained by the fact that the subset "acute-on-chronic liver failure" has a similar clinical pictureand etiopathogenetic mechanism of development withthe subset "chronic liver failure", the difference is thatthe exacerbation was caused by the appearance of aprovoking factor in a patient with chronic liver disease,which caused a similar disease dynamic. The secondformed set "Form of acute liver failure" included a subsetof "acute liver failure" and "fulminant liver failure",the latter of which is characterized by the absence ofchronic liver disease in the patient, but develops with
326
lightning speed. The third subset of "drug-induced liverinjury" only partially overlaps with the set "Form ofacute liver failure", as it has a similar clinical pictureand timing of complications, but the syndrome of hepaticencephalopathy does not develop in this event. However,in the absence of timely treatment, hepatic encephalopa-thy syndrome is manifested and the disease turns into anacute form of liver failure. Both sets represent the "Liverfailure" multiset.
The "Model of a patient with liver failure" containsgroups of criteria that ensure the exclusion of cases ofliver failure that were not considered in this study, sincethey require the interaction of specialists from differentareas of medicine. One of the groups were presentedsigns (elements of subsets), the presence of which isnecessary for the diagnosis of the liver failure form,named as inclusion criteria. The other group consisted ofnon-specific criteria, the presence or absence of which ina patient is considered necessary to clarify the clinicalpicture of the liver failure form and to establish theetiological features of occurrence of the disease in aparticular patient.
During operation, the model checks the signs fromwhich groups are present in a particular case and allow toconfirm the hypothesis about the diagnosis. On this basis,in the future, a sample was selected from the literaturefor the selection of an adequate method of treatment.
The "Liver failure treatment model" is based on adiagnostic model, taking into account various classesof treatment involving traditional and new approachesto therapy. This made it possible to form patterns thatensure the selection of analogues according to clini-cal manifestations and answers to the applied methodsand resources of treatment. Currently, this is especiallyimportant with the development of high-tech treatmentmethods. Although applied and well-proven drugs incertain situations. Thus, the main methods of treatingpatients with hepatic insufficiency are: drug therapy, livertransplantation, extracorporeal liver support systems, andcellular technologies. The choice of treatment tactics maybe affected by various symptoms, such as a history ofchronic liver disease, hepatitis, viral diseases, sepsis, etc.When analyzing the Russian and world literature, it wasfound that the principles of treatment for various formsof liver failure do not differ in regions of the world.Basically, there are similar points of view regarding thetherapeutic approach to the treatment of patients withliver failure, the use of extracorporeal technology andtransplantation, the use of stem cells.
For the formalized presentation of knowledge obtainedfrom literary sources, linguistic scales for signs of liverfailure were developed. With their help, the forms ofthe disease, described in various publications, are writtenin the form of equations, which further allows us tocarry out comparison operations with them in subsequent
analysis. The basic designations of the disease sets:∙ LF – "liver failure" multiset.∙ FCLF – "form of chronic liver failure" set.∙ FALF – "form of acute liver failure" set.∙ CLF – "chronic liver failure" subset.∙ ACLF – "acute-on-chronic liver failure" subset.∙ ALF – "acute liver failure" subset.∙ FLF – "fulminant liver failure" subset.∙ DILI – "drug-induced liver injury" subset.
The multiset, sets, and subsets consist of elements.Designations of the main elements:
∙ j – jaundice,∙ b – total bilirubin,∙ c – coagulopathy,∙ i – international normalized ratio (INR),∙ p – prothrombin activity,∙ e – hepatic encephalopathy,∙ a – ascites,∙ d – liver disease history,∙ 𝑡𝑠 – time of development of complications of liver failure
(hepatic encephalopathy and/or ascites) since the firstmanifestations of liver damage (jaundice).
The elements jaundice, total bilirubin, coagulopathy,INR, prothrombin activity, hepatic encephalopathy, as-cites, and a history of liver disease in the model areencoded in a binary type [0, 1], where 0 indicates eitherthe absence of manifestation of this trait, or, in the case oflaboratory indicators, the norm. For the time of symptomdevelopment, a scale consisting of 5 values [0, 1, 2, 3,4] was introduced, where 0 is up to 7 days (1 week), 1is up to 14 days (2 weeks), 2 is up to 28 days (4 weeks),3 – up to 56 days (8 weeks), 4 – more than 56 days (8weeks). There is a connection between some elements.For example, jaundice syndrome is diagnosed by elevatedbilirubin levels, that is:
𝑗1, 𝑏−𝑒𝑣𝑒𝑙𝑎𝑡𝑒𝑑0, 𝑏−𝑛𝑜𝑟𝑚𝑎𝑙 (1)
This statement is also true for coagulopathy syndrome andprothrombin activity and INR:
𝑐1, 𝑖≥1.5, 𝑝≤40%0, 𝑖<1.5, 𝑝>40% (2)
Using this designation of elements, the subset of "acute-on-chronic liver failure", which corresponds to the definition:"jaundice (serum bilirubin≥ 5 mg / dL) and coagulopathy (INR≥ 1.5 or prothrombin activity < 40%), with complicationsin the form of ascites and/or encephalopathy for 4 weeks inpatients with previously diagnosed or undiagnosed chronic liverdisease" [14] can be written in the following form:
⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩
𝐴𝐶𝐿𝐹 = 𝑗1, 𝑐1, 𝑎1, 𝑒1, 𝑡2𝑠, 𝑑1
𝐴𝐶𝐿𝐹 = 𝑗1, 𝑐1, 𝑎1, 𝑒0, 𝑡2𝑠, 𝑑1
𝐴𝐶𝐿𝐹 = 𝑗1, 𝑐1, 𝑎0, 𝑒1, 𝑡2𝑠, 𝑑1
𝐴𝐶𝐿𝐹 = 𝑗1, 𝑐1, 𝑎1, 𝑒1, 𝑡2𝑠, 𝑑0
𝐴𝐶𝐿𝐹 = 𝑗1, 𝑐1, 𝑎1, 𝑒0, 𝑡2𝑠, 𝑑0
𝐴𝐶𝐿𝐹 = 𝑗1, 𝑐1, 𝑎0, 𝑒1, 𝑡2𝑠, 𝑑0
(3)
that will correspond to all variants of the course of the disease,which the authors considered in their article. Similar scalesare used to record treatment methods and their results. Subse-quently, with equations, operations are performed using specialalgorithms to identify the most effective methods of treatingpatients with the same forms of the disease.
327
IV. PERSONALIZED APPROACH TO THE CHOICE OFTREATMENT
Approaches to the selection directed (targeted) [15]treatment are determined by the necessity of a per-sonalized approach to treatment with targeted areas ofaction for certain cells. This is achieved by assigningthe individual to one of the subsets of the "Model of apatient with liver failure", which are defined by the fuzzyboundaries of the various forms of this disease.
Proper selection of targeted therapy due to the needto consider the characteristics of the disease in a givenindividual. In fact, personalized treatment is selected byassigning the patient to one of the subsets of the multiset.
Clinical variants of liver failure in the form of subsets,implemented in the models described above, were usedto search for similar cases in various studies availablein the PubMed database for describing clinical caseswith parameters similar to those observed in a particularpatient.
Queries to the international publication databases re-flect the manifestations of various forms of liver failure(including exclusion terms, the combination of whichwith the underlying disease should not be containedin the collection of publications given to the doctor),treatment methods and research designs. It should benoted that in terms of evidence-based medicine, meta-analysis and randomized studies have the greatest degreeof evidence (validity). In this study, a meta-analysis wasapplied to various types of studies [16]. The formationof a database of treatment methods was carried out bymarking up the corpus of articles, including inclusioncriteria, exclusions and non-specific criteria in a modelof a patient with hepatic insufficiency.
Selection of treatment is carried out on the basis ofsimilar clinical cases, published in Russian and foreignliterary sources in relation to various ethnic groups, thatis, on cases-analogs and precedents.
V. CONCLUSION
The problem of selecting an adequate treatment for liverfailure is still relevant and causes considerable difficulties. Atthe same time, adequate therapies have appeared that providea good effect. However, the search for a method of treatmentappropriate to a particular case of disease presents a seriousproblem for the physician. Features of the clinical picture ofliver failure largely determine the tactics of patient manage-ment. A large volume of databases of literary sources does notallow timely detection of similar cases with an effective resultof treatment.
Methods of mathematical modeling, in particular the set-theoretic approach, considering fuzzy sets and subsets of formsof liver failure and suggest the selection of targeted therapyin each case using the method of meta-analysis of largespecialized databases.
REFERENCES
[1] Ch. Panackel, R. Thomas, B. Sebastian, S.K. Mathai. Recent advances inmanagement of acute liver failure. Indian J Crit Care Med, 2015, vol. 19,no 1, pp.27–33.
[2] S. Yasui, K. Fujiwara, Y. Haga, M. Nakamura, R. Mikata, M. Arai, T.Kanda, S. Oda, O. Yokosuka. Infectious complications, steroid use andtiming for emergency liver transplantation in acute liver failure: analysisin a Japanese center. J Hepatobiliary Pancreat Sci, 2016, pp.1–3.
[3] C.–T. Liu, T.–H. Chen, C.–Y. Cheng. Successful Treatment of Drug-Induced Acute Liver Failure with High-Volume Plasma Exchange. Journalof Clinical Apheresis, 2013, no 28, pp.430–434.
[4] A.B. Petrovsky. Group Multiple Criteria Decision Making: Multiset Ap-proach. Recent Developments and New Directions in Soft Computing.Studies in Fuzziness and Soft Computing, 2014, vol. 317, pp.19–33.
[5] A. Syropoulos. Mathematics of multisets. Multiset Processing, 2001, vol.2235, pp.347–358.
[6] F. Hausdorff. Set Theory. New York, AMS Chelsea Publishing, 2005, 352p.
[7] W.A.R. Weiss. An Introduction to Set Theory. CreateSpace IndependentPublishing Platform, eBook, 2014, 119 p.
[8] Z. Pawlak. Rough Sets. International Journal of Computer and InformationSciences, 1982, vol. 11, pp.341–356.
[9] V.B. Tarasov. Granuljacija informacii [Granulation information]. Podhodyk modelirovaniju myshlenija [Approach to modeling thinking]. V.G. RedkoEds. 2016, pp. 219–261.
[10] L.A. Zadeh. Fuzzy sets as a basis for a theory of possibility. Fuzzy SetsSystems, 1978, vol. 1, no 1, pp.3–28.
[11] L.A. Zadeh. Toward a theory of fuzzy information granulation and itscentrality in human reasoning and fuzzy logic. Fuzzy sets and systems,1997, vol. 90, pp.111–127.
[12] A. Bargiela, W. Pedrycz. Granular computing: an introduction. Dordrecht:Kluwer Academic Publishers, 2003.
[13] L.A. Zadeh. Rol’ mjagkih vychislenij i nechetkoj logiki v ponimanii,konstruirovanii i razvitii informacionnyh / intellektual’nyh sistem [Therole of soft computing and fuzzy logic in the understanding, design anddevelopment of information / intelligent systems]. Iskusstvennyj intellekt[Artificial Intelligence], 2001, no 2–3, pp.7–11.
[14] X.–Z. Duan, F.–F. Liu, J.–J. Tong., H.–Z. Yang, J. Chen, X.–Y. Liu, Y.–L.Mao, S.–J. Xin, J.–H. Hu. Granulocyte-colony stimulating factor therapyimproves survival in patients with hepatitis B virus–associated acute-on-chronic liver failure. World Journal of Gastroenterology, 2013, vol. 19, no7. pp.1104–1110.
[15] H.J. Kim, N. Hawke, A.S. Baldwin. NF–kappaB and IKK as therapeutictargets in cancer. Cell Death Differ, 2006, vol.13, no 5, pp.738–747.
[16] B.A. Kobrinsky, A.I. Molodchenkov, N.A. Blagosklonov, A.V. Lukin.Primenenie metodov metaanaliza v diagnostike i lechenii pacientov spechenochnoj nedostatochnost’ju [Applying meta-analysis methods in liverfailure diagnosis and treatment]. Programmnye produkty i sistemy [Soft-ware & Systems], 2017, vol. 30, no 4, pp.745–753.
ВЫБОР АНАЛОГИЧНЫХМЕТОДОВ ЛЕЧЕНИЯПЕЧЕНОЧНОЙ НЕДОСТАТОЧНОСТИ ПРИ
ИСПОЛЬЗОВАНИИТЕОРЕТИКО-МНОЖЕСТВЕННЫХМОДЕЛЕЙ
Благосклонов Н.А., Кобринский Б.А.
В статье представлены теоретико-множественные моделипеченочной недостаточности, позволяющие осуществлятьдиагностику форм заболевания и персонифицированныйподбор традиционных и новых методов лечения. Отдельныетипы представлены в виде 5 множеств, которые соответству-ют известным клиническим формам исследуемой патоло-гии: «хроническая печеночная недостаточность», «обостре-ние хронической печеночной недостаточности», «остраяпеченочная недостаточность», «фульминантная печеночнаянедостаточность» и «лекарственно-индуцированное пораже-ние печени». Модель позволяет учесть динамику переходаот одной формы к другой в процессе развития заболеванияпри продолжающемся воздействии негативных факторов.Предложенный подход учитывает различие клиническихпроявлений патологии, нечеткость переходных состоянийболезни, специфические и неспецифические признаки этио-логически и патогенетически различных форм, являющихсякритериями отбора аналогичных случаев для выбора наи-более эффективных методов терапии. Персонализированноелечение основано на сопоставлении пациента с определен-ным подмножеством мультимножества.
Received 10.01.19
328
Fuzzy logic inference ruleset augmentationwith sample data in medical decision-making
systemsAlexander Kurochkin, Vasili Sadov
Belarussian State UniversityMinsk, Belarus
[email protected], [email protected]
Abstract—Fuzzy inference systems are widely used in order toimplement complex rule-based decision-making process in expertsystems. One of their significant limitations, however, is the factthat rules themselves describe ambient semantic of decision-makingprocess without taking real world data into account. This paperdiscusses the possible ways to use sample data in order to optimizefuzzy inference-based decision making.
Keywords—expert systems, fuzzy logic, fuzzy inference systems,medical expert systems, machine learning
I. INTRODUCTION
Expert systems are a useful tool for formalizing com-plex semantic decision-making processes performed bya domain expert in order to build a software platformthat supports similar manner of decision-making. In away, creating an expert system is a means to breakdown expert knowledge and experience into a set offormal semantic logical statements that allow inductive,deductive and abductive reasoning to be applied to real-world data. This, in turn, allows these rules to reach somekinds of conclusions based on semantic representationof input data, therefore generating new knowledge orproducing new facts based on prior knowledge or existingfacts. In a way, expert systems can be viewed as higherorder formalizations of real-world data – not only dothese systems require strict semantic formalizaiton ofdata, they also require a strict semantic formalization ofany decision-making process associated with the data [1].
Inference-based expert systems use an inference en-gine as part of the decision-making process. These en-gines are expected to produce new facts based on in-trinsic expert-specified semantic rulesets and some inputfacts. Essentially, inference process defines how inputfacts and rulesets are used to generate new facts andoutput data. In order to implement an expert system givena formal inference engine it is only required to forma corresponding semantic ruleset, containing a directlogical implication path from facts that are given to theexpert system as an input to facts that can be used todetermine problem output [2], [3].
One of the significant disadvanatages of mostinference-based expert systems is the fact that onlysemantic ruels themselves serve as the ground truth for
every decision made. While this is useful when dealingwith some ambient decision-making process that cannotbe formally verified with real-world data (for instance,because the data is scarce), this is not the case most of theapplications. Usually, at least some measure of existingdata can be obtained that binds input variables and theprojected result. These data points, however, are mostlyonly used to check the correctness the decision-makingprocess, and are not used to directly improve it.
The way rule-based expert systems operate is directlyopposite to supervised machine learning approach. Insupervised machine learning, decision-making processitself is not defined in any way. Instead, learning algo-rithm is expected, given a large set of data points withknown outputs, to generate a decision-making model thatinfers data semantics and mimics the intrinsic input-output relationship. This approach yields great resultswhen the direct semantic relationship between input andoutput parameters exists, but is not obvious; this semanticrelationship can be expected to be determined during thelearning process itself. However, the exact reasoning thatled a fully-trained supervised machine learning modelto produce specific output based on specific input isusually hard or impossible to determine, i.e. the decision-making process itself remains a black box; moreover, thisdecision-making process is expected to be different fordifferent learning models and input data, the differencesometimes being very significant [4].
The aim of this paper is to discuss the possible waysto combine these two approaches – to employ super-vised machine learning techniques in order to optimizeand augment the expert system ruleset and its variableparameters to better fit existing sample data. The pro-cess is applied to fuzzy inference-based decision-makingmedical expert system for determining chorionicity.
II. EXPERT SYSTEMS WITH FUZZY INFERENCE INMEDICAL APPLICATIONS
A key component of any expert system is inferenceengine. It defines, precisely, in what form expert knowl-edge should be presented to the system in order to
329
support further decision-making process, i.e. it defines aformalization of the expert knowledge. Most commonly,inference engines require decision-making process to bedescribed as a number of rules that are assumed to betruthful logical statements and can be utilized to generatenew facts about existing data.
In general, the ruels used in inference engine aredefined as a number of if-then rules, i.e. “IF (A) THEN(B)”. Logical statement A (antecedent) is usually someknown fact, and logical statement B (consequent) is a factthat can be naturally deduced from A. Inference engineswork by applying existing knowledge A to produce newknowledge B, i.e. by asserting the truth of consequentsbased on antecedent across the rulesets. The fact thatconsequent is true can be used as an output fact, i.e.as something that must be determined in context of theproblem solved by expert system, or as prior knowledgefor another rule, i.e. as part of antecedent statementfor other rules, which, in turn, allows to generate morefacts based on these rules with modus ponens inference.The process of cascading application of existing facts isknown as chaining.
Most of the known inference engines employ ei-ther forward chaining or backward chaining in orderto deduce the result. The primary difference betweenthe two is the form of input to the system. Forwardchaining means that problem input can be stated as anumber of facts that are then used as antecedents inthe ruleset, and the goal of the system is to deducesome of the consequent facts. Backward chaining meansthat problem input is not only a number of facts usedin antecedents, but also a formal logical statement, andthe goal of the system is to determine, based on giveninput facts, whether this statement is true according toexisting ruleset. In general, backward chaining is harderto implement, but it allows expert systems to work asknowledge bases and use these input formal statements asa queries, implicitly performing a semantic analysis withexisting facts. Such systems require non-linear traversalmechanism for rulesets in order to deduce which rulesexatcly must be applied in order to verify the givenstatement. Forward chaining is usually easier to reasonabout, since it is only applied to deduce the correct factsbased on known data provided as an input [2], [5].
One of the common approaches to building inferenceengines in expert systems is usage of fuzzy logic-basedinference. Based on fuzzy algebra, fuzzy inference allowsinput facts to be stated not as a strict logical statements,but as fuzzy variables, i.e. with a specific degree ofcertainty. It enables a one-way formalization of concreteinput parameter values to statements as fuzzy sets inorder to form a fuzzy variable, that can be further usedin logical statements of a ruleset to generate new data.
In a fuzzy inference system (FIS) any kind of numericparameter can be used as input or output variable. From a
higher order of applicaiton, FIS is a function - providinga projection of input parameters into output parameters.Input parameters are variables that are known prior, andoutput parameters are the unknown variables that mustbe determined by applying a set of rules.
When specifying rules in FIS, the elementary logicalexpressions used in them are generally defined not as astrict boolean value that can be true or false, but ratheras a fuzzy variable that can have an arbitary degree of“truth-iness” from 0 to 1. In order to simplify the rulegeneration, all input variables can have several fuzzy setsassociated with them that can be used as rules [3].
For instance, a medical expert system for determiningchorionicity and amnionicity in multiple pregnancies canhave a set of rules stated by the expert [6], [7]. One ofthem might look like this:
IF [(amniotic membrane thickness) IS “Thick”]AND [(duration) IS “End of the 1-st trimester”]
THEN [(chorionicity) IS “Likely dichorional”](1)
The rule (1) has 2 input variables (amniotic mem-brane thickness and duration) in antecedent and 1output variable (chorionicity) in consequent. The an-tecedent is a compound statement - it has two sim-pler logical terms grouped with conjunction (logical“AND”). The first operand of conjunction is a statement[(amniotic membrane thickness) IS “Thick”]. In fuzzyinference systems all input and output parameters areassumed to have a crisp numeric value. For instance, oneof the possible values for amniotic membrane thicknessis 2.1mm, as measured during ultrasound inspection. Butit is usually hard to operate with crisp values in the rules,because the terms in them allow for a degree of uncer-tainty. Here, we use the term “Thick”. By itself, the state-ment [(amniotic membrane thickness) IS “Thick”] ac-tually defines a fuzzy set over crisp tickness values, inmilimeters, which means that, for any crisp thicknessvalue, it is possible to determine the “truth-iness” mea-sure of this statement. For instance, for value of 2.1mmthat statement might yield a 0.95 certainty fuzzy value.The exact mapping from crisp values to fuzzy certainityfor each of the terms used in the rule is determined by afuzzy membership function. Likewise, duration variablewith the term “End of the 1-st trimester” is also a fuzzyset over the duration of the pregnancy, in weeks [6], [7].
The membership functions are usually selected froma range of basic functions, like triangular, trapezoid,sigmoidal, etc. The parameters of these functions canregulate slope, translation points and the general shapeof the function. Typically, the parameters for membershipfunctions are also defined by the expert. For instance, thefact amniotic membrane thickness is considered “Thick”at >2mm is part of the expert knowledge that is for-malized within the system. The "gray" areas, however,where membership function takes values between 0 and1, are usually interpolated between a known set of points
330
linearly. For example, trapezoid membership functionwould be a common choice for “Thick” term, becausethere is a certainty that thickness over 2mm can beconsidered thick and below 1.7mm cannot be consideredthick, so between 1.7mm and 2mm the values are linearlyor sigmoidally interpolated.
Fuzzy inference process aggregates all the logicalexpressions in atecedents using a specific implementationof t-norm and t-conorm for AND and OR operators.Based on the output membership, all consequents thatcontain a specific output variable can be assigned aspecific value. In classic Mamdani inference type system,resulting output values of the system are fuzzy sets. Inorder to produce a crisp value based on the fuzzy set, acentroid is most commonly used [2].
III. FUZZY LOGIC SYSTEM PARAMETERS ASMACHINE LEARNING OPTIMIZATION VARIABLES
As noted earlier, expert systems in general and fuzzylogic inference systems in particular only rely on theirrespective rulesets with a formal decision-making pro-cess in order to produce output values, and a significantdisadvantage of such an approach is the fact that realdatasets of the problem cannot be used to improvedecision-making process. On the other hand, re-creatinga decision-making process from scratch based on realdata is the problem that machine learning aims to solve[1], [4].
In general, machine learning works based on theassumption that output data (the result of the decision-making process, in our case) can be modelled as aparemetric function:
~y =M(~x, ~θ), (2)
where ~y is the output parameter vector, ~x is the inputparameter vector and ~θ is the model parameters vector.Given a number of data points with known input andoutput (a training set) (~xi, ~yi), it is possible to calculatehow well the model (2) fits these data points. The mostcommonly used metric is sum of squared differences(SSD) defined as follows:
S(~θ) =
n∑
i=1
(M(~xi, ~θ)− ~yi)2 (3)
Other metrics that can be used include sum of absolutedifferences and mean values of squared error and abso-lute error, and other estimators based on median values.The general idea is that this metric can be described as“fitness”, i.e. it numerically represents the ability of aparticular model M with a concrete set of parameters ~θto generalize these data points [1].
The learning process itself is essentially an optimizingof any model performance metric, like (3), in regards tomodel parameters ~θ. The general idea is that variableparameters of the model ~θ can be adjusted to make the
model M behave in every possible way, and of thosepossible functions those are best used for a particularproblem that guarantee that performance metric is at itsminimum across all possible values of model parameters.
More complex models tend to fit the data better.However, the complexity of the model itself means thatit performs poorly in generalization – additional non-linearity introduced into it works well for exact datapoints used for training, but the function itself can behaveunpredictably in between these points. This problemis known as overfitting. If the model is overfit, itsperformance on the training set will be very good, but itwill perform poorly on any known data items not usedduring training, and requires additional optimizations likeparameter regularization [1].
As described earlier, machine learning, while able tofit the data for a variety of tasks, actually remains ablack box even when properly trained. The decision-making process within, for instance, neural networks isbased entirely on individual weight values. Sometimesit is possible to analyze the paths from input to outputand determine how input features affect the output;however, most of the times this information doesn’t shedlight on how exactly the decisions are made. This isthe main difference between classic expert systems andmachine learning systems – the former work based ona formalized decision-making process without taking thereal data into account, while the latter try to determinea set of parameters for some complex function so that itfits the real data without trying to formalize the decision-making proecss [4], [8].
Combinining fuzzy logic and neural networks to obtainthe benifits of both have been explored in the pastwith the introduction of ANFIS (Adaptive Neuro-FuzzyInference System). However, the complexity of suchsystems mean that they remain a universal estimator andthat their decision-making process is still obscure, sinceit depends heavily on training set [8]–[10].
In medical systems, real data for training is usuallyavailable in smaller quantities. At the same time, doctorsthat observe and diagnose the patients usually followa set of generalist rules that aid with decision-making.These rules can be formed based on personal experience,or observing historical data, or taken from a well-knownresearch on the topic, but the preference is usually givento methodology rather than statistics. For this reason,expert systems are a more natural choice for medicalapplications. However, historical data can and should beused not only to verify built systems, but also to helpimprove them. The radical improvement would mean thecomplete reorganization of the rulesets if, for instance,some of the rules can be disproven by a certain case.However, such decisions must be carefully weighted bythe expert himself, since outliers in medical practice area common occurance [6], [7].
331
The above means that incorporating real-world datastatistical distribution as a basis for decision-makingprocess, as is done in machine learning, is generally nota desireable approach. As such, a better way to use realdata is to make smaller adjustments to the formal rule-based expert systems [10].
The formal parameters that can be optimized are ruleweights and membership function values. It is importantto note that both weights and membership function pa-rameters must be constrained, because those parametersare part of the expert knowledge formalization, as well.
IV. OPTIMIZING MEDICAL FIS FOR DETERMININGCHORIONICITY WITH HISTORICAL DATA
Given a ruleset of k rules, rule weight vector ~w of length kdetermines the scaling factor of this particular rule. The outputmembership of antecedent of i-th during antecedent aggregationare additionally multiplied by wi.
Given a trapezoid membership function mtrap with param-eters a1 < a2 < a3 < a4 defined as follows:
mtrap(x) =
0, if x ∈ (−∞; a1] ∪ [a4;∞)x−a1a2−a1
, if x ∈ (a1, a2)
1, if x ∈ [a2; a3]a4−xa4−a3
, if x ∈ (a3, a4)
(4)
It is possible to treat a1 - a4 as model parameters per fuzzymembership, i.e. for every term of every input and outputvariable in the model. The outlines of membership functions(4) should still be defined by the expert, so it’s usually helpfulto formalize them as a set of predefined constraints a(max)
i
and a(min)i for these parameters and allow data optimization
to variate parameters within these constraints.A parameter vector for optimization ~θ consists, as such, of
rule weights wi for every rule and trapezoid membership func-tion parameters aij that lie within their respective constraintsa(max)ij and a(min)
ij for every term membership.The only optimization strategy for such an algorithm is
an iterative mesh descent, since function gradients cannot beapproximated. This algorithm provides a very low guaranteeof finding global minimum, but with existing decision-makingmodel local minima already provide better results, as shownin the table I. The training set included 300 cases with knownchorionicity with 10% used for cross-validation. Input param-eters include results of various laboratory and examinationreviews, based on which an expert was asked to provide aresolution. The results indicate that pure machine learningapproach requires further tuning or model selection as it retainslarger error in outlier cases, while fuzzy logic with optimiza-tion generally performs better than pure fuzzy inference, aspredicted, approaching the expert estimation errors.
Table ITRAINING AND VALIDATION ERRORS FOR ALL PREDICTIONS, BY
DIFFERENT APPROACH
Approach Training error Validation errorExpert estimation 8.4% N/AFeedforward ANN 11.4% 19.6%
FIS 13.3% N/AFIS with optimization 9.3% 12.8%
V. CONCLUSION
Optimizing expert systems based on real-world data is apowerful way not only to verify the formal decision-makingmodel, but to also augment it with statistical observations.This allows to retain the clarity of formal decision process, asformulated by an expert, while allowing the outliers present inlive data to also be reflected in the model in a form of weightsand membership function parameters. The results indicate thatthis approach yields a noticeable accuracy increase for fuzzyinference systems. Further studies, however, are required inorder to optimize the augmentation process, since the only wayof determining correct variable parameters is a full traversal,making it a costly and time-consuming process.
REFERENCES
[1] I. H. Witten, E. Frank, M. A. Hall et al Data Mining: PracticalMachine Learning Tools and Techniques, 4th ed.. Burlington,Morgan Kaufmann, 2016. 654 p.
[2] X. Wang, D. Nauck, M. Spott, R. Kruse. Intelligent data analysiswith fuzzy decision trees. Soft Computing, 2007, vol. 11, no. 5,pp. 439-457.
[3] L. A. Zadeh Fuzzy sets. Information and Control, 1965, vol. 8,no. 3, pp. 338-353.
[4] R. A. Ghani, S. Abdullah, R. Yaakob Comparisons betweenartificial neural networks and fuzzy logic models in forecastinggeneral examinations results. International Conference on Com-puter, Communications, and Control Technology (I4CT), 2015,pp. 253-257.
[5] J. R. Quinlan Induction of Decision Trees. Machine Learning,1986, vol. 1 pp. 81-106.
[6] O. Pribushenya, A. Kurochkin Otsenka platsentatsii pri mnogo-plodnoi beremennosti s ispol’zovaniem sovremennykh ekspert-nykh komp’yuternykh programm [Placentation evaluation ofmultiple pregnancies using modern expert computer programs].Sovremennye perinatal’nye meditsinskie tekhnologii v resheniiproblem demograficheskoi bezopasnosti [Modern prenatal med-ical technologies in solving demographic safety problems], 2017,vol. 10, pp. 106-111.
[7] A. Kurochkin, O. Pribushenya, V. Sadov Ekspertnaya meditsin-skaya sistema po opredeleniyu khorial’nosti na osnove sistemynechetkoi logiki [Expert medical system for chorionicity evalua-tion based on fuzzy logic system]. Informatsionnye tekhnologii isistemy [Information technologies and systems], 2017, pp. 92-93.
[8] S. Rajab Handling interpretability issues in ANFIS using rule basesimplification and constrained learning. Fuzzy Sets and Systems,2018, in press.
[9] R. Mazouni, A. Rahmoun AGGE: A Novel Method to Auto-matically Generate Rule Induction Classifiers Using GrammaticalEvolution. Studies in Computational Intelligence, 2015, vol. 570,pp. 270-288.
[10] E. Hüllermeier Does machine learning need fuzzy logic? FuzzySets and Systems, 2015, vol. 281, pp. 292-299.
УТОЧНЕНИЕ НАБОРА ПРАВИЛ СИСТЕМЫНЕЧЕТКОГО ВЫВОДА С ИСПОЛЬЗОВАНИЕМИСТОРИЧЕСКИХ ДАННЫХ ВМЕДИЦИНСКИХ
СИСТЕМАХ ПОДДЕРЖКИ ПРИНЯТИЯ РЕШЕНИЙКурочкин А.В., Садов В.С.
Нечеткий вывод – широко распространённый подход кпостроению сложных процессов принятия решений в экс-пертных системах на базе набора правил. Одним из суще-ственных недостатков таких систем является самостоятель-ное описание семантики процесса вывода без учета реальныхданных. В работе рассматриваются возможные способыиспользования существующих исторических данных с цельюоптимизации процесса принятия решений с использованиемнечеткого вывода.
Received 09.01.19
332
Do Intellectual Systems Need Emotions?Maksim Davydov,
Anatoly OsipovBelarusian State University
of Informatics and RadioelectronicsMinsk, Belarus
[email protected], [email protected]
Sergei KilinB.I.Stepanov Institute of Physics
National Academyof Sciences of Belarus
Minsk, [email protected]
Vladimir KulchitskyInstitute of Physiology
National Academyof Sciences of Belarus
Minsk, [email protected]
Abstract—One of the unresolved issues in the problemof artificial intelligence is the question of the desirabilityof introducing elements of motivation and emotions intoartificial neural networks. The attitude to this issue is stillcontroversial. The coding system in the interfaces is basedon the binary system of the "on-off" type. In the casethat an emotional component is included into the artificialintelligence system, it is advisable to introduce, along withbinary coding system, a positive or negative attitude to thedecision which is being made by implementing a feedbacksystem. Thus, the “on” or “off” due to the feedback can, ata certain level of “noise” or emotional disturbances, declineor turn into appropriate opposite. This article presents anattempt to critically understand the existing problem ofan effective result of artificial intelligence in the presenceof positive or negative emotional component of varyingintensity and duration. It is important that all eventsassociated with artificial intelligence functioning should beconsidered not as static ones, but as events developingsimultaneously in time and space.
Keywords—intellectual systems, feedback, emotions, nat-ural neural networks
I. INTRODUCTION
Artificial intelligence got its name due to the factthat researchers have been trying to create a system thatfragmentarily uses the architecture of natural intelligenceand, this way, approximates in its capabilities the naturalintelligence [1]. Since intelligence is often associatedwith human abilities in the field of creativity, when cre-ating such system it is necessary to take into account asmany of the constituent elements, presenting attributes ofthe creative realities of natural intelligence, as possible.One of these elements is emotional component of thenatural intelligence [2-7]. Experts discuss the feasibilityof the component, but in reality it is not widely usedin modern models– of the artificial intelligence. Letus give some examples from everyday life to confirmthis idea. The role of the emotional state of poets,for example, Pushkin and Virgil, when writing goodverses in the morning, after sleeping, is well known. Theexamples above confirm the importance of the emotionalcomponent in the formation of the final result in theactivities of the neural networks of the brain and thewhole organism. In the previously published article bythe authors of this paper, this component was actually
schematically indicated only [8]. In this report, we willtry to understand the existing problem in more detail.
II. “TO BE OR NOT TO BE”, DO INTELLECTUALSYSTEMS NEED EMOTIONS?
Marvin Minsky believes that emotional cycles leadto long fixation, thus reproducing a contrasting emo-tional state (joy-grief) [1]. From this point of view, theaddition of the emotional component to the system ofartificial intelligence will inevitably be accompanied bythe formation of an external reaction of the intellectualsystem, which indicates the relation of the system to thedecision made [9-13]. That is, in fact, it is possible todetermine the success or failure of the task performed,by considering the external behavior of the artificialintelligence system.
Experts in the field of artificial intelligence, who createcomputer architecture to reproduce the human brain inthe form of chips and interfaces, build the boards whichconsist of 16 million neurons as, for instance, the boardmade by the experts from IBM. For comparison, thetotal number of neurons in the human brain amountsfrom 80 to 100 billion. Thus, additional development oftechnologies is required to ensure more realistic consis-tency between the artificial intelligence and the naturalone. These examples demonstrate the importance of theemotional component in the functioning of both naturaland artificial neural networks [14-16].
Further. If, in the process of artificial intelligencecreation the option of complete analogy of the artificialintelligence with the natural intelligence has been cho-sen, then we arrive at well-known fact that depression,depressed state of a person is often accompanied by acomplete lack of efficiency and, in fact, by the blockadeof functioning of neural networks responsible for theefficiency of performing complex tasks. While in case ofmodel experiments such effects are interesting and validfor the analysis of final and sometimes fatal outcomes,the introduction of the emotional component into com-plex systems of artificial intelligence that are responsiblefor (critical processes) vehicle movement, rocket launch,piloting airplanes, etc. can lead to fatal results. For better
333
understanding the essence of the problem, let us turnto the structure of the functional system by Pyotr K.Anokhin.
We note that the multi-element feedback scheme byPyotr K. Anokhin [17-20] has an emotional component.The presence of feedback allows the neural networks ofthe brain to correct the signals coming from the brainto the executive organs in order to improve the actionperformed. In case of any deviation of the feedbacksignals from the pattern of command signals that initiatethe execution of the action, adaptive changes occur inneural networks that control the effectiveness of theaction performed. The result of these changes is theformation of a new pattern of command signals thatprovide more accurate implementation of the intendedaction. When the pattern of the conceived action andthe pattern of the performed one are fully coincide, thefunctional system stops functioning as the planned resulthas been achieved.
Such coincidence of the conceived and performedactions is accompanied by the formation of positiveemotions. In natural neural networks, this principle ofoperation is the basis for effective achievement of the re-sult, when positive solution of the task is achieved withina short time and with minimum power consumption. Onthe other hand, the occurrence of additional “noises” inthe functional system, the source of which are negativeemotions, reduces the efficiency of the neural networkfunctioning, since part of the neurons of the network,and sometimes the entire network, are simultaneouslyinvolved in performing at least two or several tasks.
The introduction of the emotional component intothe system of artificial intelligence will inevitably beaccompanied by additional noise caused by the process-ing of the “emotional” component and the assessment(consideration) of its role in making the final decision.For the time being, there is no clear answer to thisquestion because of the relatively small number of workson this subject. But a priori, it is assumed that additionalnoise in neural networks that control the execution of anyaction will introduce additional interference in achievingthe “final adaptive result”.
Figure 1 shows in schematic form a variant of the func-tioning of the neural networks of the brain stem, whichensure the maintenance of vital respiratory function. Thefigure shows highlighted chemoreceptor link, which wasformed in the process of evolution of the neuroglialcells of the brain (Figure 1). Brain stem chemoreceptorsdo not respond to hypoxia, but to a miniature shiftof hydrogen ions in the brain tissue. Such mechanismof self-regulation of brain functions primarily ensurescontrol of the intensity of metabolic processes in thebrain, during which, even with normal amount of oxygenin the brain, acidosis is often formed in the brain tissuedue to disruption of the reverse chemoreceptor function,
which maintains the optimal amount of hydrogen ions(pH), the content of CO2 as well as O2.
Figure 1. Regulation of lung ventilation (scheme). 1 - carotid body, 2- medullary (central) chemoreceptors, 3 - respiratory center structures,4 - vagal afferent nerve terminals, 5 - diaphragm. XII - the nucleus ofthe hypoglossal nerve. Arrows correspond to directions of signal flow(figure by Dmitry Tokalchik [25]).
Figure 2 demonstrates in vitro relationships in a net-work of neurons, which are the basis for the formation ofreactions of neurons of one network when the functionalstate of neurons of another network changes. This figureshows more than 20 neuron-like elements with clearcontours of processes. Dominated cells of 25-35 microns.There are single cells of small size of about 10 microns,which are also involved in the formation of a neuralnetwork.
As we know, the functional state of local neuralnetworks has a direct impact on the pattern of totalbrain activity [4, 7, 18, 21]. This, in turn, determines theemotional state of a living organism [17]. Also, the emo-tional background is influenced by various chemicals:neurotransmitters, hormones and others. Neurotransmit-ters have a direct impact on the state of neural networksdue to the fact that they transmit an electrochemicalimpulse from a nerve cell through the synaptic spacebetween neurons. The release into the intercellular spaceof various mediators can inhibit or accelerate the workof individual neurons. And the work of a single neuronmodulates the functional state of the neural network.Another feature of neurotransmitters is that they can havea selective effect. So there are neurons that have receptorsfor serotonin (serotonergic neurons), adrenaline (adren-ergic neurons), dopamine (dopaminergic neurons) andothers on the membrane. As a rule, one neuron is able to
334
Figure 2. Formation of a neural network in an in vitro experiment.
perceive only one neurotransmitter. Thus, a certain typeof neurotransmitter modulates the work of a specific partof the neural network. For example, serotonin is oftencalled the “hormone of happiness,” and dopamine is animportant part of the reward system. This is possible dueto the fact that these neurotransmitters modulate the workof the neural networks of the brain, which are involvedin the formation of a certain type of emotion: joy,satisfaction. It is not a secret for anybody that a person’sworking capacity is connected with his emotional state.In an excellent mood, work is done faster, the efficiencyof solving complex and non-standard tasks increases, andfatigue decreases. With a negative emotional background,in addition to a decrease in performance, problemsmay arise with social communication. In severe cases,pathological conditions develop: depression, symptomsof various mental illnesses. For example, the dysfunctionof dopamine regulation of neural networks leads todisruptions in the functioning of the reward system. Asa result, the person ceases to enjoy the work done, thetask at hand, meeting with friends, tasty food and otherpleasant events. In addition to the obvious changes in theemotional state of breakdown in dopamine regulation, theprocesses of learning and motivation are disturbed. It isnatural that in this case a person has big problems.
Summarizing the above, a set of factors of differentnature is involved in maintaining the normal operationof the neural network. These include the principles oforganization and architecture of a neural network, thenumber of neurons and the methods of their synapticcontacting with each other (chemical, electrical, mixed).Also an important aspect in evaluating the neural networkarchitecture is the structures (Figure 2) that are involvedin the formation of synapses (axo-dendritic, axo-somatic,axo-axonal and dendro-dendritic synapses). This allows
modulating the processing paths of information and thepoint of its entry into the neural network. It is also im-portant to consider the environment in which the neuralnetwork elements are located. It is necessary to take intoaccount not only the physicochemical parameters thatensure the vital activity and maintenance of the shapeof the neural network and its components (Figure 1).It is also important to control and manage the balanceof bioactive molecules, in particular, neurotransmitters.This will allow selectively activating and modulating thework of specific neurons and launch directed chemicalcascades.
III. CONCLUSION
Just few articles on the intellectual approach for adap-tive control of a nonlinear dynamic system, includingemotional training of the brain, are available [16]. Theresults demonstrated not only excellent improvement inthe actions performed, but also smaller energy consumedby the dynamic system. There are more areas of applica-tion of artificial intelligence systems with an “emotionalcomponent”. The World Health Organization report for2017 noted that almost 5% of the world’s populationsuffers from a deep depression, which is accompaniedby a decrease in the quality of life. Psychotherapy doesnot provide any solution to this growing global publichealth problem. Technologies based on the artificial in-telligence and evidence in interactive mobile applicationscan play a role in filling this gap. It is clear that suchtechnologies will not replace health care professionals tomore seriously address mental health problems. However,application technologies can act as an additional orintermediate support system [21-24].
335
ACKNOWLEDGMENT
The authors gratefully acknowledge the assistance ofProf. V. Golenkov.
REFERENCES
[1] Minsky M. The emotion machine: Commonsense thinking, artifi-cial intelligence, and the future of the human mind. – Simon andSchuster, 2007.
[2] Martınez-Miranda, Juan, and Arantza Aldea. "Emotions in humanand artificial intelligence." Computers in Human Behavior 21.2(2005): 323-341.
[3] DeLancey, Craig. Passionate engines: What emotions reveal aboutthe mind and artificial intelligence. Oxford University Press,2001.
[4] Yang Y.X., Gao Z.K., Wang X.M., Li Y.L., Han J.W., MarwanN., Kurths J. A recurrence quantification analysis-based channel-frequency convolutional neural network for emotion recognitionfrom EEG. Chaos. 2018, 28(8): 085724. doi: 10.1063/1.5023857.
[5] Shibata, Takanori, Makoto Yoshida, and Junji Yamato. "Artifi-cial emotional creature for human-machine interaction." IEEEinternational conference on systems man and cybernetics. Vol.3. Institute of electrical engineers INC (IEEE), 1997.
[6] Picard, Rosalind W., Elias Vyzas, and Jennifer Healey. "Towardmachine emotional intelligence: Analysis of affective physiolog-ical state." IEEE transactions on pattern analysis and machineintelligence 23.10 (2001): 1175-1191.
[7] Shibata, Takanori, Kazuyoshi Inoue, and Robert Irie. "Emotionalrobot for intelligent system-artificial emotional creature project."Robot and Human Communication, 1996., 5th IEEE InternationalWorkshop on. IEEE, 1996.
[8] Davydov M.V., Osipov A.N., Kilin S.Y., Kulchitsky V.A. NeuralNetwork Structures: Current and Future States // Open semantictechnologies for intelligent systems. OSTIS-2018. P.259-264.
[9] Bringsjord, Selmer, and David Ferrucci. Artificial intelligenceand literary creativity: Inside the mind of brutus, a storytellingmachine. Psychology Press, 1999.
[10] Ramos, Carlos, Juan Carlos Augusto, and Daniel Shapiro. "Am-bient intelligence—the next step for artificial intelligence." IEEEIntelligent Systems 23.2 (2008): 15-18.
[11] Cohen, Paul R., and Edward A. Feigenbaum, eds. The handbookof artificial intelligence. Vol. 3. Butterworth-Heinemann, 2014.
[12] Wilson, Elizabeth A. Affect and artificial intelligence. Universityof Washington Press, 2011.
[13] Jendoubi T., Strimmer K. A whitening approach to probabilisticcanonical correlation analysis for omics data integration. BMCBioinformatics. 20(1):15. 2019
[14] Maksimenko V.A., Hramov A.E., Frolov N.S., Lüttjohann A.,Nedaivozov V.O., Grubov V.V., Runnova A.E., Makarov V.V.,Kurths J., Pisarchik A.N. Increasing Human Performance bySharing Cognitive Load Using Brain-to-Brain Interface. FrontNeurosci. 2018.
[15] Michal, Ptaszynski, et al. "Towards context aware emotionalintelligence in machines: computing contextual appropriatenessof affective states." Proceedings of the Twenty-First InternationalJoint Conference on Artificial Intelligence (IJCAI-09). AAAI,2009.
[16] Bellmund J.L.S., Gärdenfors P., Moser E.I., Doeller C.F. Nav-igating cognition: Spatial codes for human thinking. Science.362(6415). 2018.
[17] Anokhin P.K. Nodular mechanism of functional systems as a self-regulating apparatus. Prog. Brain Res. 1968. 22:230-251.
[18] Anokhin P.K. Electroencephalographic analysis of corticosubcor-tical relations in positive and negative conditioned reactions. AnnN Y Acad Sci. 1961. 92:899-938.
[19] Sadeghieh A., Sazgar H., Goodarzi K., Lucas C. Identificationand real-time position control of a servo-hydraulic rotary actuatorby means of a neurobiologically motivated algorithm. ISA Trans.2012. Vol. 51(1): 208-219. doi: 10.1016/j.isatra.2011.09.006.
[20] Inkster B., Sarda S., Subramanian V. An Empathy-Driven, Con-versational Artificial Intelligence Agent (Wysa) for Digital MentalWell-Being: Real-World Data Evaluation Mixed-Methods Study.JMIR Mhealth Uhealth Vol. 6(11). 2018.
[21] Torres F., Puente S.T., Úbeda A. Assistance Robotics andBiosensors. Sensors (Basel). 2018. 18(10). pii: E3502. doi:10.3390/s18103502.
[22] Servick K. Brain scientists dive into deep neural net-works. Science. 2018. 361(6408):1177. doi: 10.1126/sci-ence.361.6408.1177.
[23] Cavazza M. A Motivational Model of BCI-Controlled HeuristicSearch. Brain Sci. 8(9). 2018.
[24] Zappacosta S., Mannella F., Mirolli M., Baldassarre G. Generaldifferential Hebbian learning: Capturing temporal relations be-tween events in neural networks and the brain. PLoS ComputBiol. 2018. 14(8):e1006227. doi: 10.1371/journal.pcbi.1006227.eCollection 2018 Aug.
[25] Kulchitsky V, Zamaro A, Koulchitsky S. Hypoxia and Hyper-capnia: Sensory and Effector Mechanisms. Biomedical Journalof Scientific & Technical Research. 2018 Vol.8 (4): 1-3. DOI:10.2671/BJSTR.2018.08.001692.
НУЖНЫ ЛИ ИНТЕЛЛЕКТУАЛЬНЫМСИСТЕМАМ ЭМОЦИИ?
Давыдов М. В., Осипов А. Н.,Килин С. Я, Кульчицкий В. А.
Вопрос о необходимости введения элементов мотива-ции и эмоций в нейронные сети – одна из нерешенныхпроблем в области искусственного интеллекта. К даннойпроблеме до сих пор относятся противоречиво. Системакодирования в интерфейсах основана на двоичной системетипа включено/выключено. В случае, когда в интеллек-туальную систему включается эмоциональный компонент,рекомендуется помимо двоичной системы кодирования вво-дить также положительное или отрицательное отношениек принимаемому решению путем реализации механизмаобратной связи. Таким образом, под влиянием обратнойсвязи состояние "включено"или "выключено"может приопределенном уровне "шума"или под воздействием эмоци-ональных расстройств поменяться на противоположное. Вданной статье предпринимается попытка критически осмыс-лить существующую проблему функционирования системискусственного интеллекта при наличии положительногоили отрицательного эмоциональных компонентов различ-ной степени интенсивности и продолжительности. Важнопонимать, что все события, связанные с функционированиемсистем искусственного интеллекта должны рассматриватьсяне как статические, а как развивающиеся одновременно впространстве и времени.
Received 10.01.19
336
Algorithm of generation finite element meshfor the system «vertebrae – intervertebral disk
– vertebrae» based on the stl modelKonstantin Kurochka, Kosntantin Panarin, Ekaterina Karabchikova
Gomel State Technical University named after P.SukhojGomel, Belarus
[email protected], [email protected], [email protected]
Abstract—The authors propose an algorithm for gener-ating a regular finite-element mesh, based on the three-dimensional «vertebrae-intervertebral disk-vertebrae» stlmodel, that resistant to defects in the model, which al-lows the model to be correctly partitioned if there areintersecting objects or objects with incompletely touchingsurfaces. The resulting finite elements of different objectswill be guaranteed to match, that is, the sides and edgesof neighboring elements will be guaranteed to repeat eachother.
Keywords—Mesh, finite elements, stress-strain state,modeling, vertebra, stl
I. INTRODUCTION
In the study of the state of the lumbar spine of aperson, very often there is the problem of identifyingareas of the spine that are exposed to the greatest loads.This will allow to choose the best treatment methodand, as a result, minimize the rehabilitation period. Inaddition, when choosing a surgical procedure, it is oftennecessary to estimate how the stress-strain state of thevertebra changes, how the surgery affects the stress-strainstate of other vertebrae, and how the stress-strain stateof the lumbar spine changes over time after surgery [1],[2].
To estimate the state of the lumbar spine usually usedradiographic methods, which make it possible to get avisual picture of the current state of the spine [3] (Fig. 1).
Figure 1. An example of radiography (a) and computer tomography(b) images
But radiographic pictures do not allow us to estimatethe stress-strain state or predict the development of theresulting stresses over time.
Thus, when treating a patient and examining the stateof his lumbar spine, necessary to solve a number ofproblems presented in Fig. 2
Figure 2. The general scheme of studying the state of the lumbar spine
To generate STL files based on the results of CTimages obtained in the DICOM format, an algorithmbased on vertebral search and localization [4] was used.
To determine the stress-strain state of mechanical andbiomechanical systems, in practice very often the finiteelement method [5] is used, which allows one to finddeformations from the action of different loads on anycomplex non-uniform three-dimensional objects.
II. EXISTING SOLUTIONS TO THE PROBLEM
When using the finite element method, the initialstage is the approximation of the studied area and itsdiscretization into grid cells (finite elements). This stageis one of the key and most significant ones, as furthercalculations are completely based on the obtained resultsof discretization the source domain. Due to the variousfeatures of the structure, shape and size of the stud-ied three-dimensional models, an individual approachis required for the correct execution of the discretiza-tion stage, except for cases with simple models. Finiteelement meshes can be separated into regular (fig. 3a), containing objects of the same shape and size, andirregular (fig. 3 b), varying the size of the elements formaximum match the original shape [6].
When discretizating objects of complex shape on anirregular grid, there may be places where finite elementsacquire an irregular shape, an uneven size of the sides
337
Figure 3. Regular (a) and non-regular (b) finite element mesh
(the size of one side greatly exceeds the others) or thefinite element becomes too small. As a consequence ofthese problems, the calculation becomes impossible, orthe accuracy of the calculation results based on a suchmodel decreases.
In this paper, the model is a «vertebrae – intervertebraldisk – vertebrae» system, which is determined by a stl-file with a three-dimensional model (Fig. 4) describingthe surfaces of vertebrae and intervertebral disks.
Figure 4. Example of stl-model «vertebrae - intervertebral disc -vertebrae»
On the basis of this model, the construction of a finiteelement mesh is carried out. This file was formed as aresult of reconstruction of the spinal column accordingto the results of image analysis obtained using computedtomography and X-ray, in connection with which theobjects of the stl-model have uneven edges, and may notcompletely touch the surfaces or intersect and overlap.Described factors impose limitations for partitioning themodel into finite elements by standard means.
Existing software for mesh creating, such as NetGen[7] and TetGen [8], allows to customize various param-eters of a finite element mesh, but when constructinga mesh, they produce a discretization based on theiralgorithms not only on the internal area of a three-dimensional object, but also on its surface. The resultof approximation of the surface by finite elements is its
simplifications and distortion, in consequence of whichbetween the vertebrae and intervertebral discs are formedthe intersection of surfaces or gaps with the lack ofdirect contact, which results the impossibility of furthercalculations.
Using Gmsh software [9] allows you to avoid thisproblem, since this software performs the discretizationof the internal area of the object based on the existingsurface points. The resulting finite element meshes havethe original correctly touching surfaces, without gapsand intersections. However, Gmsh makes it impossible tocorrectly split into finite elements wide and flat objects,such as an intervertebral disk. With a small distancebetween the upper and lower planes, the applicationperforms a separation by connecting the points of theseplanes directly, which leads to the appearance of asmall number of finite elements whose dimensions ofthe sides are comparable to the size of the entire disk.The presence of such elements makes it impossible tocorrectly calculate the stress-strain state of the spine.
III. DEFECT-RESISTANT REGULAR MESHPARTITIONING ALGORITHM
The authors propose an algorithm for constructing aregular finite-element mesh based on a three-dimensionalmodel of the human spine, that allow to avoid intersec-tions of the boundaries of elements of various objects,resistant to arbitrary irregularities of the surfaces ofmodel objects, defects of objects, as well as gaps betweenobjects. The application of the proposed algorithm to thestl-file allows you to get a regular finite element meshfor the entire model.
The algorithm for constructing a finite element meshcan be separated into the following steps:
• the establishment of maximum and minimum valuesof the length, width and height of the analyzedmodel and the formation of the corresponding re-gion of space;
• dividing the resulting region of space into a finite setof regular parallelepiped cells by forming verticaland horizontal lines — constructing a regular grid;
• finding the set of points of intersection of the formedstraight lines with the elements of the objects of theanalyzed three-dimensional model;
• removal of parallelepiped cells that are not relatedto any of the objects contained in the model;
• the division of parallelepipeds into finite elements(tetrahedra);
• correlation of the resulting tetrahedra with the cor-responding model objects.
The first step of the algorithm is to establish a workingregion of space in which the model is considered. Thisarea is a parallelepiped wrapping the model, to obtain thedimensions of which, the points (vertices) of the consid-
338
ered model are searched for, which have the smallest andlargest values for each of the coordinate axes.
The second stage of the algorithm is based on the con-cept of regular (structured) meshes, according to whichthe model under consideration is completely placed inthe resulting region, on which the grid is subsequentlysuperimposed. This mesh is formed by constructing a setof straight lines defined by a fixed step in three directionsof coordinate axes. In this case, the step in each of thethree directions may differ from each other, which allowsto vary the shape and size of the final finite elements.
In other words, at this stage determined by a pluralityof straight lines that define a set of ribs parallelepipedsserving as the basis for the further formation of the finiteelement.
At the third stage, the whole set of triangular elementsincluded in the three-dimensional stl-model presentedis traversed in order to find the points of intersectionwith the previously constructed set of lines forming aregular mesh. The points of intersection of lines withtriangles obtained as a result of this stage are fixed,thereby forming a set of segments located inside theregion defined by the initial stl-model.
As a result of the analysis of the obtained segmentsin the fourth stage, a set of parallelepipeds is com-piled, which are guaranteed to be part of the three-dimensional model under consideration. Parallelepipeds,in the construction of which the obtained segments werenot involved, are discarded.
Then each parallelepiped is discretized into 6 equaltetrahedra. For each tetrahedron checked affiliation ofits model and determines which element of the lumbarspine it belongs. Depending on the element belonging, itsphysical and mechanical characteristics (elastic modulusand Poisson’s ratio) are determined.
A total of 4 different structural elements are considered[10]:
• the cortical tissue;• the intervertebral disk;• the spongy tissue;• the transverse processes.An example of a finished finite element mesh is shown
in Fig. 5With such partition into finite elements boundary
planes of different objects are guaranteed to be thesame, i.e. the part of parallelepipeds forming neighboringfinite elements and their eedges obtained as a resultof sampling will be guaranteed to repeat each other,which would eliminate the possibility of the empty spacebetween the elements and the intersection boundaries ofobjects.
The regular mesh obtained as a result of applying thealgorithm makes it possible to perform a to perform finiteelement modeling of stress-strain state model of the hu-man spine, the results of which can then be used to solve
Figure 5. The results of the finite element mesh generation for a systemof «vertebrae – intervertebral disc – vertebrae» based on stl-model.
such problems as determining the risk and predicting thecourse of diseases of the musculoskeletal system, andselecting the optimal treatment strategy before surgeryand evaluation of the effectiveness of measures taken inthe treatment process.
IV. VERIFICATION OF THE PROPOSEDDECOMPOSITION ALGORITHM
To verify the finite element mesh generated by theproposed algorithm and verify its suitability for solvingreal problems of stress-strain modeling, the simulationresults in FreeFem software using the built-in irregu-lar mesh generator were compared with the simulationresults based on the mesh generated by the proposedalgorithm.
As the first test case was considered a cylinder with aradius of 11 mm. and a height of 90 mm from the samematerial. Applied vertical load is concentrated at the topcenter of the cylinder plane. Fig. 6 presents the obtaineddisplacement values for both meshes.
It can be seen from the graph that the displacementvalues for the resulting mesh and for the mesh from theFreeFem program are almost identical, the error does notexceed 3%.
V. FINDING THE STRESS-STRAIN STATE OF THELUMBAR SPINE
The proposed algorithm was tested on a real modelof the human lumbar spine, reconstructed on the basisof medical images. The model contained two groups ofelements with different material characteristics - L1-L5vertebrae and intervertebral disks.
The material of the vertebrae is bone, possessing thefollowing physical characteristics:
• Young’s modulus 3432 MPa• Poisson’s ratio of 0.3• Density of the fabric is 2020 kg/m3
339
Figure 6. Comparison of the results of the calculation of the test modelwith the cylinder using its own mesh and mesh of FreeFem software
The material of the intervertebral disc has the follow-ing physical characteristics:
• Young’s modulus 559 MPa• Poisson’s ratio of 0.4• Density 1090.3 kg/m3
A vertical uniformly distributed load is applied tothe upper plane of the vertebra L1. Fig. 7 presents theobtained values of displacements in this model.
Figure 7. Graph of displacement in the lumbar spine model
From the graph one can see a clear dependence of thedisplacement values on the material of the object — asharp change in displacements occurs at the border ofintervertebral disks.
VI. CONCLUSION
Algorithm for constructing regular finite element meshbased on the three-dimensional stl-model «vertebrae- intervertebral disc - vertebrae» was developed. Thealgorithm is resistant to arbitrary irregularities of thesurfaces of model objects, defects in objects and gapsbetween objects, which allows applying it to models ofany quality.
The correctness of the algorithm is verified by compar-ing the simulation results based on the grid generated bythe algorithm with the simulation results in the FreeFemsoftware package.
REFERENCES
[1] E.L. Tsitko, A.F. Smeyanovich, E.S. Astapovich, E.V. Tsitko,RRentgenometricheskii analiz kinematiki L4 - L5 i L5 - S1pozvonochnykh segmentov v III stadii degenerativnogo protsessa,Novosti Khirurgii, 2015, vol. 23, pp. 202–208.
[2] C. Xiong, A. Suzuki, M.D. Daubs, T. Scott, K. Phan, J.Wang, Theevaluation of cervical spine mobility without significant spondy-losis by kMRI, European Spine Journal, 2015, vol. 24(12), pp.2799-2806.
[3] E.L. Tsitko, K.S. Kurochka, N.N. Masalitina, I.N. Tsalko, V.V.Komrakov, E.V. Tsitko, Rentgenometricheskaya otsenka kine-matiki poyasnichno-kresttsovogo otdela pozvonochnika pri os-teokhondroze s pomoshch’yu programmnogo sredstva «Volot»,Problemy zdorov’ya i ekologii, 2017, vol. 4 (54), pp. 35–41.
[4] K. S. Kurochka, T. V. Luchsheva, K. A. Panarin, Lokalizatsiyapozvonkov cheloveka na rentgenovskikh izobrazheniyakh s is-pol’zovaniem Darknet YOLO, Minsk, Doklady BGUIR, 2018, vol.3 (113). pp. 32–38.
[5] J. N. Reddy, An Introduction to the Finite Element Method, NewYork, McGraw-Hill Education, 2006, p. 784.
[6] Zhangxin Chen, The Finite Element Method: Its Fundamentals andApplications in Engineering, Singapore, World Scientific Publish-ing, 2011, p. 348.
[7] J. Schöberl, NETGEN An advancing front 2D/3D-mesh generatorbased on abstract rules, Computing and visualization in science,1997, vol. 1, pp. 41–52.
[8] H. Si, TetGen. Tetrahedral mesh generator and threedimensionaldelaunay triangulator, ACM Transactions on Mathematical Soft-ware, 2015, vol. 41 (2), pp. 1–36.
[9] C. Geuzaine, J.F. Remacle, Gmsh: A 3-D finite element mesh gen-erator with built-in pre- and post-processing facilities, InternationalJournal for Numerical Methods in Engineering, 2009, vol. 79 (11),pp. 1309–1331.
[10] B.A. Christiansen, D.L. Kopperdahl, D.P. Kiel, T.M. Keaveny,M. L. Bouxsein, Mechanical contributions of the cortical andtrabecular compartments contribute to differences in age-relatedchanges in vertebral body strength in men and women assessed byQCT-based finite element analysis, Journal of Bone and MineralResearch, 2011, vop. 26(5), pp. 974–983.
АЛГОРИТМ ГЕНЕРАЦИИКОНЕЧНОЭЛЕМЕНТНОЙ СЕТКИ ДЛЯ
СИСТЕМЫ «ПОЗВОНОК –МЕЖПОЗВОНОЧНЫЙ ДИСК – ПОЗВОНОК»
НА ОСНОВЕ STL МОДЕЛИ
Курочка К. С., Панарин К. А., Карабчикова Е. А.
Авторами предлагается устойчивый к дефектаммодели алгоритм генерации регулярной конечноэле-ментной сетки на основе трехмерной stl-модели «по-звонок – межпозвоночный диск – позвонок», позво-ляющий производить корректное разбиение моделипри наличии пересекающихся объектов или объектовс не польностью соприкосающимися поверхностями.Получаемые конечные элементы различных объектовбудут гарантированно совпадать, т.е стороны и ребрасоседних элементов будут гарантированно повторятьдруг друга.
Received 30.12.18340
AUTHOR INDEX
A
Alekseev Alexey 277
Al-Haji S. 311
Al-Masri A. 311
Azarov Elias 103
B
Baria Andrii 155
Blagosklonov Nikolay 325
Bogatyrev Vladimir 315
Borgest Nikolai 165
Buhaienko Yurii 151
Burdo Georgy 243
C
Chaiko Yelena 257
D
Davydenko Irina 25, 53
Davydov Maksim 333
Dolbin Alexey 293
Dolganovskaya
Aleksandra 123
Dorodnykh Nikita 179
E
Egereva Irina 197
Emelyanova Irina 197
Eremeev Aleksandr 25, 201
Erofeev Aleksandr 205
F
Fedorischev Leonid 21
Filimonova Ekaterina 285
Filippov Aleksey 123
Fomenkov Sergey 293
G
Gladun Anatoly 161
Globa Larysa 151, 155
Golenkov Vladimir 25, 53
Golovko Vladimir 91, 215
Gorshkov Sergey 183
Grabusts Peter 145
Grakova Natalia 53, 215
Gribova Valeria 21
Grigoricheva Maria 123
Guliakina Natalia 25, 201
Guskov Gleb 123
H
Halavataya Katsiaryna 269
Hardzei Aliaksandr 281
Hubarevich Nastassia 103
I
341
Ishchenko Ivan 151
Iskra Natallia 113
Iskra Vitali 113
Ivaniuk Dzmitry 91, 215
Ivashenko Valerian 91, 247
K
Karabchikova Ekaterina 337
Kasyanik Valery 215
Kilin Sergei 333
Kleschev Alexander 21
Kobrinskii Boris 325
Kolesnikov Alexander 133, 139
Kolomoitcev Vladimir 315
Koroleva Maria 243
Kovalev Mikhail 91
Kozhukhov Alexander 201
Krapivin Yury 289
Krasnoproshin Viktor 265
Kroshchanka Aliaksandr 91
Kulchitsky Vladimir 333
Kurachka Kanstantsin 273, 337
Kurochkin Alexander 329
L
Liashenko Andrii 151
Listopad Sergey 133
Lobanov Boris 297
Lukashevich Marina 113
Lyahor Timofei 103
M
Malochkina Anastasiya 165
Massel Alexey 209
Massel Liudmila 209
Matskevich Vadim 265
Moroz Anastasiia 155
Moskalenko Philip 21
Mukhitova Aigul 173
N
Nestsiarenia Ihar 273
O
Orlova Yulia 277, 293
Osipov Anatoly 333
P
Paliukh Boris 197
Panarin Kosntantin 337
Petrovsky Alexey 277
Pilipchuk Andrei 261
Pilipchuk Ludmila 261
Polyachok Eugene 261
Polyakov Vladimir 315
Polyakova Irina 285
342
Prihozhy Anatoly 305
Prokhorenko Vladislav 251
R
Rogushina Julia 161
Romanov Anton 123
Rovbo Maksim 237
Rozaliev Vladimir 277, 293
Rusetski Kirill 215
Rybina Galina 129
S
Sadov Vasili 269, 329
Samodumkin Sergei 231
Shalfeeva Elena 21
Shaya B. 311
Shebalov Roman 183
Shunkevich Daniil 53, 215
Smorodin Viktor 251
Solochshenko Alexandr 257
Soloviev Sergey 285
Sorokin Dima 129
Sorokin Ilya 129
T
Taberko Valery 91, 215
Taranchuk Valery 225
Tarassov Valery 187
Timchenko Vadim 21
Tynarbay Marzhan 321
U
Udovichenko Anna 281
Ulyev Andrey 277
V
Vetrov Alexander 197
Vishniakou U. 311
Vitulyova Yelizaveta 257
Vorobiev Vitaly 237
Y
Yarushkina Nadezhda 123
Yurin Alexander 179
Z
Zaboleeva-Zotova Alla 277
Zahariev Vadim 103
Zbrishchak Svetlana 169
Zhdanouski Arseni 305
Zhitko Vladimir 297
Zhizhimov Oleg 173
Zianouka Evgeniya 301
Zorina Tatyana 209
343
АВТОРСКИЙ УКАЗАТЕЛЬ
А
Азаров И. С. 103
Алексеев А. В. 277
Б
Баря А. Д. 155
Благосклонов Н. А. 325
Богатырев В. А. 315
Боргест Н. М. 165
Бугаенко Ю. М. 151
Бурдо Г. Б. 243
В
Ветров А. Н. 197
Витулёва Е. С. 257
Вишняков В. А. 311
Воробьев В. В. 237
Г
Гладун А. Я. 161
Глоба Л. С. 151, 155
Голенков В. В. 25, 53
Головатая Е. А. 269
Головко В. А. 91, 215
Гордей А. Н. 281
Горшков С. В. 183
Грабуст П. С. 145
Гракова Н. В. 53, 215
Грибова В. В. 21
Григоричева М. С. 123
Губаревич А. В. 103
Гулякина Н. А. 25, 201
Гуськов Г. Ю. 123
Д
Давыденко И. Т. 25, 53
Давыдов М. В. 333
Долбин А. В. 293
Долгановская А. Ю. 123
Дородных Н. О. 179
Е
Егерева И. А. 197
Емельянова И. И. 197
Еремеев А. П. 25, 201
Ерофеев А. А. 205
Ж
Ждановский А. М. 305
Жижимов О. Л. 173
Житко В. А. 297
З
Заболеева-Зотова А. В. 277
Захарьев В. А. 103
344
Збрищак С. Г. 169
Зеновко Е. С. 301
Зорина Т. Г. 209
И
Иванюк Д. С. 91, 215
Ивашенко В. П. 91, 247
Искра В. В. 113
Искра Н. А. 113
Ищенко И. А. 151
К
Карабчикова Е. А. 337
Касьяник В. В. 215
Килин С. Я 333
Клещев А. С. 21
Кобринский Б. А. 325
Ковалев М. В. 91
Кожухов А. А. 201
Колесников А. В. 133, 139
Коломойцев В. С. 315
Королева М. Н. 243
Крапивин Ю. Б. 289
Краснопрошин В. В. 265
Крощенко А. А. 91
Кульчицкий В. А. 333
Курочка К. С. 273, 337
Курочкин А. В. 329
Л
Листопад С. В. 133
Лобанов Б. М. 297
Лукашевич М. М. 113
Ляхор Т. В. 103
Ляшенко А. В. 151
М
Малочкина А. В. 165
Массель А. Г. 209
Массель Л. В. 209
Мацкевич В. В. 265
Мороз А. М. 155
Москаленко Ф. М. 21
Мухитова А. 173
Н
Нестереня И. Г. 273
О
Орлова Ю. А. 277, 293
Осипов А. Н. 333
П
Палюх Б. В. 197
Панарин К. А. 337
Петровский А. Б. 277
Пилипчук А. С. 261
345
Пилипчук Л. А. 261
Поляков В. И. 315
Полякова И. Н. 285
Полячок Е. Н. 261
Прихожий А. А. 305
Прохоренко В. А. 251
Р
Ровбо М. А. 237
Рогушина Ю. В. 161
Розалиев В. Л. 277, 293
Романов А. А. 123
Русецкий К. В. 215
Рыбина Г. В. 129
С
Садов В. С. 269, 329
Самодумкин С. А. 231
Смородин В. С. 251
Соловьев С. Ю. 285
Солощенко А. В. 257
Сорокин Д. О. 129
Сорокин И. А. 129
Т
Таберко В. В. 91, 215
Таранчук В. Б. 225
Тарасов В. Б. 187
Тимченко В. А. 21
Тынарбай М. 321
У
Удовиченко А. М. 281
Ульев А. Д. 277
Ф
Федорищев Л. А. 21
Филимонова Е. А. 285
Филиппов А. А. 123
Фоменков С. А. 293
Ч
Чайко Е. В. 257
Ш
Шайя Б. Х. 311
Шалфеева Е. А. 21
Шебалов Р. Ю. 183
Шункевич Д. В. 53, 215
Э
Эль Масри А. Х. 311
Эль Хаджи С. К. 311
Ю
Юрин А. Ю. 179
Я
Ярушкина Н. Г. 123
346
Научное издание
Открытые семантические технологии
проектирования интеллектуальных систем
Open Semantic Technologies
for Intelligent Systems
СБОРНИК НАУЧНЫХ ТРУДОВ
Основан в 2017 году
Выпуск 3
В авторской редакции
Ответственный за выпуск В. В. Голенков
Компьютерная вёрстка Н. В. Гракова, А. В. Губаревич
Подписано в печать 30.01.2018. Формат 60×84 1/8. Бумага офсетная. Гарнитура «Таймс».
Отпечатано на ризографе. Усл. печ. л. 46,5. Уч.-изд. л. 66,4. Тираж 140 экз. Заказ 19.
Издатель и полиграфическое исполнение: учреждение образования
«Белорусский государственный университет информатики и радиоэлектроники».
Свидетельство о государственной регистрации издателя, изготовителя,
распространителя печатных изданий 1/238 от 24.03.2014,
2/113 от 07.04.2014, 3/615 от 07.04.2014.
ЛП 02330/264 от 14.04.2014.
220013, Минск, П. Бровки, 6
10th international scientific and technical conference
«Open Semantic Technologies
for Intelligent Systems»
Open Semantic Technologies for Intelligent Systems OSTIS-
2020 February 20-22, 2020 Minsk. Republic of Belarus
C A L L F O R P A P E R S
We invite you to take part in X International Scientific and Technical Conference “Open
Semantic Technologies for Intelligent Systems” (OSTIS-2020), which will focus on areas of use of the
semantic technologies.
Conference will take place from February, 20th to February, 22nd, 2020 at the Belarusian
State University of Informatics and Radioelectronics, Minsk, Republic of Belarus.
Conference proceedings language: English
Working languages of the conference: Russian, Belarusian, English
MAIN ORGANIZERS OF THE CONFERENCE Russian Association for Artificial Intelligence (RAAI)
Belarusian State University of Informatics and Radioelectronics (BSUIR) State Institution “Administration of High Technologies Park” (Republic of Belarus) Ministry of Education Ministry of Communications and Informatization
CONFERENCE TOPICS:
Underlying principles of semantics-based knowledge representation, and their unification.
Types of knowledge and peculiarities of the semantics-based representation of various knowledge and
metaknowledge types.
Links between knowledge; relations, that are defined on the knowledge.
Semantic structure of a global knowledge base, that integrates various accumulated knowledge.
Parallel-oriented programming languages for processing of the semantics-based representation of knowledge bases.
Models for problem solving, that are based on knowledge processing, which occurs directly at the semantics-based
representation level of knowledge being processed. Semantic models of information retrieval, knowledge integration,
correctness and quality analysis of knowledge bases, garbage collection, knowledge base optimization, deductive and
inductive inference in knowledge bases, plausible reasoning, pattern recognition, intelligent control. Integration of
various models for problem solving
Semantic models of environment information perception and its translation into the knowledge base.
Semantic models of multimodal user interfaces of intelligent systems, based on the semantic representation of
knowledge used by them, and unification of such models.
Semantic models of natural language user interfaces of intelligent systems. The structure of semantic representation
of linguistic knowledge bases, which describe natural languages and facilitate solution of natural language text and
speech interpretation problems, and of natural language texts and speech messages synthesis, that are semantically
equal to certain knowledge base fragments.
Integrated logic-semantic models of intelligent systems, based on semantic knowledge representation, and their
unification
Various technical platforms and implementation variants of unified logic-semantic models of intelligent systems,
based on semantic knowledge representation
Models and means, that are based on the semantic representation of knowledge and that are oriented to the design of
various typical components of intelligent systems (knowledge bases, programs, problem solvers, user interfaces).
Models and means, that are based on semantic representation of knowledge and that are oriented to the complex
design of various classes of intelligent systems (intelligent reference systems, intelligent learning systems, intelligent
control systems, intelligent robotics systems, intelligent systems for design support etc.)
Applied intelligent systems, that are based on the semantic representation of knowledge used by them
CONFERENCE GOALS AND FORMAT
The goal of the conference is to discuss problems of creation of the Open Complex Semantic
Technology for Intelligent Systems Design. This determines the Conference format, which involves
(1) plenary reports; (2) workshops; (3) round tables, dedicated to discussion of various questions of
creating of such technology; (4) poster sessions.
During the poster sessions every participant of the conference will have an opportunity to
demonstrate his results. Conference format assumes exact start time of each report, and exact time of
its exhibition presentation.
One of the major objectives of the conference is to attract not only scientists and postgraduate
students, but also students who are interested in artificial intelligence, as well as commercial
organizations willing to collaborate with research groups working on the development of modern
technologies for intelligent systems design.
PARTICIPATION TERMS AND CONDITIONS
All those interested in artificial intelligence problems, as well as commercial organizations
willing to collaborate with research groups working on the development of modern technologies for
intelligent systems design are invited to take part in the Conference.
To participate in the OSTIS-2020 conference, it is necessary to register in the CMT system
before December 15, 2019, find conference page, and from there:
submit a participation form for the OSTIS-2020 conference. Each participation form field is
required, including indication of the reporter. By filling in the registration form, you agree that
your personal data will be processed by the Organizing Committee of the Conference, and that
the paper and information about the authors will be published in printed and electronic format.
Participation form should contain information on all of the authors. If author(s) are participating
with a report, participation form should have their color photo(s) attached (they are needed for
the Conference Program);
upload an article for publication in the OSTIS-2020 Conference Proceedings. Papers should be
formatted according to the provided template (see http://proc.ostis.net/eng/autors.html). Four
full pages is a minimum size of a paper.
send the signed scan of the letter of consent
If a report is submitted to participate in one of the contests, this intention should be clearly
indicated in the participation form.
The selection of papers for publication in the Conference Proceedings and participation in the
Conference is performed by a number of reviewers from among the members of the Conference
Program Committee.
Non-compliant applications and papers will be rejected.
Authors, whose articles were included in the Conference Program, will receive the invitations
for participating in the Conference before January 30th, 2020.
Conference participation does not require any fees.
PAPERS SUBMISSION PROCEDURE
Papers (only on topics mentioned above) should be submitted ready for publication
(http://proc.ostis.net/eng/main.html -> For authors). The text should be logically complete and contain
new scientific and practical results. Each author is allowed to submit a maximum of two reports.
After the article was submitted, it is sent for review. Review results will become available to the
paper author(s) on the CMT website before January 25th.
The Organizing Committee reserves the right to reject any paper, if it does not meet the
formatting requirements and the Conference topics, as well as if there was no participation form
submitted for the paper.
YOUNG SCIENTIST REPORTS CONTEST
Authors of the report submitted to the contest may include scientists with scientific degrees, but
the report should be made by those without a degree and under 35 years old.
To take part in the young scientists report contest, it is necessary to:
1) fill in the participation form, where your participation in the contest is clearly indicated;
2) write an article and upload it to the CMT website;
3) fill in, sign, scan and send letter of consent via the email.
4) make a report at the conference (in person);
YOUNG SCIENTIST PROJECTS CONTEST
Projects of applied intelligent systems and systems aimed at supporting the design of intelligent
systems are allowed to take part in the contest; they have to be presented by a scientist without a
degree and under 35 years old.
To take part in the young scientist projects contest, it is necessary to:
1) fill in the participation form, where your participation in the contest is clearly indicated;
2) write an article and upload it to the CMT website;
3) make a report at the conference (in person);
4) make an exhibition presentation of the software
STUDENT INTELLIGENT SYSTEM PROJECTS CONTEST
To participate in the contest, a project must meet the following criteria: (a) it was developed by
students and/or undergraduates of the higher education institutions, and (b) project consultants and
advisors must hold a scientific degree and title. To participate in this contest, it is necessary to:
1) familiarize yourself with contest's terms and conditions (http://www.conf.ostis.net);
2) fill in the participation form for the contest (http://www.conf.ostis.net);
3) prepare a summary of the project (http://www.conf.ostis.net).
4) submit the participation form and project summary to the student projects' email address:
RESEARCH PAPERS COLLECTION PUBLICATION
The Conference Organizing Committee plans to publish the papers selected by the Program
Committee based on the results of their review, in the Research Papers Collection, on the official
Conference website http://conf.ostis.net , and on the Research Papers Collection website
http://proc.ostis.net.
Upon successful review author sends a letter of consent to the Organizational Committee.
Author therefore agrees that his paper can be made freely available in electronic form at other
resources at the Editorial Board's discretion.
KEY DATES OF THE CONFERENCE
October 1st, 2019 paper submission opens
December 15th, 2019
January 25th, 2020
paper submission deadline
paper review deadline
January 20th, 2020 final decision on paper publication; sending out invitations and
notifications on inclusion of a paper in the OSTIS-2020 Scientific works
collection
February 10th, 2020 Draft Conference Program publication on the conference website
http://conf.ostis.net
February 15th, 2020 Research Papers Collection and Conference Program publication on the
Research Papers Collection website http://proc.ostis.net
February 20st, 2020 Participant registration and OSTIS-2020 conference opening
February 20st to
22rd, 2020
OSTIS-2020 conference
February 26, 2020 Photo report and conference report publication on the conference website:
http://conf.ostis.net
March 15th, 2020 Research papers collection will be uploaded to the Russian Science
Citation Index database
CONFERENCE PROGRAM FORMATION
Conference program is formed by the Program Committee according to the paper review
results; author(s)' confirmation of participation is required as well.
CONTACTS
All the necessary information about the forthcoming and previous OSTIS Conferences can be
found on the conference website http://conf.ostis.net and http://proc.ostis.net.
For questions regarding conference participation and dispute resolution please contact:
Methodological and advisory support to the conference participants shall be provided through
the conference e-mail only.
The conference venue is the 5th academic building of the Belarusian State University of
Informatics and Radioelectronics (39, Platonovа str., Minsk, Republic of Belarus)
X международная научно-техническая конференция
«Открытые семантические технологии
проектирования интеллектуальных систем»
Open Semantic Technologies for Intelligent Systems OSTIS-
2020 20 – 22 февраля 2020 г. Минск. Республика Беларусь
И Н Ф О Р М А Ц И О Н Н О Е П И С Ь М О
Приглашаем принять участие в X Международной научно-технической конференции
«Открытые семантические технологии проектирования интеллектуальных систем» (OSTIS-
2020), которая будет посвящена вопросам области применения семантических технологий.
Конференция пройдет в период с 20 по 22 февраля 2020 года в Белорусском
государственном университете информатики и радиоэлектроники, г. Минск, Республика
Беларусь.
Язык статей сборника конференции: английский
Рабочие языки конференции: русский, белорусский, английский.
ОСНОВНЫЕ ОРГАНИЗАТОРЫ КОНФЕРЕНЦИИ Российская ассоциация искусственного интеллекта (РАИИ)
Белорусский государственный университет информатики и радиоэлектроники (БГУИР) Государственное учреждение «Администрация Парка высоких технологий» (Республика Беларусь) Министерство образования Республики Беларусь Министерство связи и информатизации Республики Беларусь
НАПРАВЛЕНИЯ РАБОТЫ КОНФЕРЕНЦИИ:
Принципы, лежащие в основе семантического представления знаний, и их унификация.
Типология знаний и особенности семантического представления различного вида знаний и метазнаний.
Связи между знаниями и отношения, заданные на множестве знаний.
Семантическая структура глобальной базы знаний, интегрирующей различные накапливаемые знания
Языки программирования, ориентированные на параллельную обработку семантического представления баз
знаний
Модели решения задач, в основе которых лежит обработка знаний, осуществляемая непосредственно на
уровне семантического представления обрабатываемых знаний. Семантические модели информационного
поиска, интеграции знаний, анализа корректности и качества баз знаний, сборки информационного мусора,
оптимизации баз знаний, дедуктивного и индуктивного вывода в базах знаний, правдоподобных
рассуждений, распознавания образов, интеллектуального управления. Интеграция различных моделей
решения задач
Семантические модели восприятия информации о внешней среде и отображения этой информации в базу
знаний
Семантические модели мультимодальных пользовательских интерфейсов интеллектуальных систем, в
основе которых лежит семантическое представление используемых ими знаний, и унификация этих моделей
Семантические модели естественно-языковых пользовательских интерфейсов интеллектуальных систем.
Структура семантического представления лингвистических баз знаний, описывающих естественные языки
и обеспечивающих решение задач понимания естественно-языковых текстов и речевых сообщений, а также
задач синтеза естественно-языковых текстов и речевых сообщений, семантически эквивалентных
заданным фрагментам баз знаний
Интегрированные комплексные логико-семантические модели интеллектуальных систем, основанные на
семантическом представлении знаний, и их унификация
Различные технические платформы и варианты реализации интерпретаторов унифицированных логико-
семантических моделей интеллектуальных систем, основанных на семантическом представлении знаний
Средства и методы, основанные на семантическом представлении знаний и ориентированные на
проектирование различных типовых компонентов интеллектуальных систем (баз знаний, программ,
решателей задач, интерфейсов)
Средства и методы, основанные на семантическом представлении знаний и ориентированные на
комплексное проектирование различных классов интеллектуальных систем (интеллектуальных справочных
систем, интеллектуальных обучающих систем, интеллектуальных систем управления, интеллектуальных
робототехнических систем, интеллектуальных систем поддержки проектирования и др.)
Прикладные интеллектуальные системы, основанные на семантическом представлении используемых ими
знаний
ЦЕЛЬ И ФОРМАТ ПРОВЕДЕНИЯ КОНФЕРЕНЦИИ
Целью конференции является обсуждение проблем создания открытой комплексной
семантической технологии компонентного проектирования интеллектуальных систем.
Этим определяется и формат её проведения, предполагающий (1) пленарные доклады, (2)
секционные заседания; (3) круглые столы, посвященные обсуждению различных вопросов
создания указанной технологии; (4) выставочные презентации докладов.
Выставочная презентация докладов даёт возможность каждому докладчику
продемонстрировать результаты своей разработки на выставке. Формат проведения
конференции предполагает точное время начала каждого доклада и точное время его
выставочной презентации.
Важнейшей задачей конференции является привлечение к её работе не только учёных и
аспирантов, но и студенческой молодежи, интересующейся проблемами искусственного
интеллекта, а также коммерческих организаций, готовых сотрудничать с научными
коллективами, работающими над интеллектуальными системами и созданием современных
технологий и их проектированием.
УСЛОВИЯ УЧАСТИЯ В КОНФЕРЕНЦИИ
В конференции имеют право участвовать все те, кто интересуется проблемами
искусственного интеллекта, а также коммерческие организации, готовые сотрудничать с
научными коллективами, работающими над созданием современных технологий
проектирования интеллектуальных систем.
Для участия в конференции OSTIS-2020 необходимо до 15 декабря 2020 года
зарегистрироваться в системе CMT, найти страницу конференции и на ней:
подать заявку на конференцию OSTIS-2020. Каждое поле заявки обязательно для
заполнения, в том числе указание того автора, кто будет представлять доклад. Заполняя
регистрационную форму, Вы подтверждаете согласие на обработку Оргкомитетом
конференции персональных данных, публикацию статей и информации об авторах в
печатном и электронном виде. В заявке должна содержаться информация по каждому
автору. К заявке доклада должны быть прикреплены цветные фотографии всех авторов
статьи (это необходимо для публикации Программы конференции);
загрузить статью для публикации в Сборнике материалов конференции OSTIS-2020.
Статья на конференцию должна быть оформлена в соответствии с правилами
оформления публикуемых материалов и занимать не менее 4 полностью заполненных
страниц;
загрузить сканированный вариант письма о согласии на публикацию и размещения
передаваемых материалов в сети Интернет.
Если доклад представляется на конкурс докладов молодых учёных или на конкурс
программных продуктов молодых учёных, это должно быть явно указано в заявке доклада.
Отбор статей для публикации в Сборнике и участия в работе конференции
осуществляется рецензентами и редакционной коллегией сборника.
Заявки и статьи, оформленные без соблюдения предъявляемых требований, не
рассматриваются.
До 30 января 2020 года, авторам статей, включённых в Программу конференции,
направляются приглашения для участия в конференции.
Участие в конференции не предполагает организационного взноса.
ПОРЯДОК ПРЕДСТАВЛЕНИЯ НАУЧНЫХ СТАТЕЙ
Статьи (только по перечисленным выше направлениям) представляются в готовом для
публикации виде (http://proc.ostis.net -> Авторам). Текст статьи должен быть логически
законченным и содержать новые научные и практические результаты. От одного автора
допускается не более двух статей.
После получения статьи, она отправляется на рецензирование и в срок до 25 января на
сайте CMT вы сможете ознакомиться с результатами рецензирования
Оргкомитет оставляет за собой право отказать в приеме статьи в случае, если статья не
будет соответствовать требованиям оформления и тематике конференции, а также, если будет
отсутствовать заявка доклада, соответствующая этой статье.
КОНКУРС ДОКЛАДОВ МОЛОДЫХ УЧЁНЫХ
Среди авторов доклада, представляемого на конкурс докладов молодых учёных, могут
входить учёные со степенями и званиями, но непосредственно представлять доклад должны
авторы, не имеющие степеней и званий в возрасте до 35 лет.
Для того, чтобы принять участие в конкурсе научных докладов молодых учёных
необходимо:
1) заполнить заявку на участие в конференции, в которой чётко указать своё желание принять
участие в данном конкурсе;
2) написать статью на конференцию и загрузить на сайте CMT;
3) заполнить, подписать, отсканировать и отправить по почте письмо о согласии;
4) лично представить доклад на конференции.
КОНКУРС ПРОЕКТОВ МОЛОДЫХ УЧЁНЫХ
Принимать участие в конкурсе проектов молодых учёных могут проекты прикладных
интеллектуальных систем и систем, ориентированных на поддержку проектирования
интеллектуальных систем, при этом представлять проект на конкурсе должен молодой учёный
в возрасте до 30 лет, не имеющие учёных степеней.
Для того, чтобы принять участие в конкурсе программных продуктов молодых учёных
необходимо:
1) заполнить заявку на участие в конференции), в которой чётко указать своё желание принять
участие в данном конкурсе;
2) написать статью на конференцию и загрузить на сайте CMT;
3) лично представить доклад на конференции;
4) провести выставочную презентацию, разработанного программного продукта.
КОНКУРС СТУДЕНЧЕСКИХ ПРОЕКТОВ ИНТЕЛЛЕКТУАЛЬНЫХ СИСТЕМ
В конкурсе студенческих проектов могут принимать участие проекты, разработчиками
которых являются студенты и магистранты высших учебных заведений, консультантами и
руководителями проекта могут быть лица, имеющие научную степень и звание. Для того,
чтобы принять участие в данном конкурсе необходимо:
1) ознакомиться с положением о конкурсе студенческих проектов (http://www.conf.ostis.net ->
Конкурсы -> Конкурс студенческих проектов интеллектуальных систем);
2) заполнить заявку на участие в конкурсе студенческих проектов (http://www.conf.ostis.net ->
Конкурсы -> Конкурс студенческих проектов интеллектуальных систем);
3) подготовить описание проекта (http://www.conf.ostis.net -> Конкурсы -> Конкурс
студенческих проектов интеллектуальных систем).
4) выслать заявку на участие в конкурсе и описание проекта по электронному адресу конкурса
студенческих проектов: [email protected].
ПУБЛИКАЦИЯ МАТЕРИАЛОВ КОНФЕРЕНЦИИ
Оргкомитет конференции предполагает публикацию статей, отобранных Программным
комитетом по результатам их рецензирования, в Сборнике материалов конференции и на
официальном сайте конференции http://conf.ostis.net и официальном сайте сборника
http://proc.ostis.net.
По результатам рецензирования автор отправляет оргкомитету письмо о согласии,
которое предусматривает дальнейшую возможность размещения статей, вошедших в сборник
конференции, в открытом электронном доступе на иных ресурсах по усмотрению редакции
сборника.
КЛЮЧЕВЫЕ ДАТЫ КОНФЕРЕНЦИИ
1 октября 2019г. начало подачи материалов для участия в конференции
15 декабря 2019г.
25 января 2020г.
срок получения материалов для участия в конференции Оргкомитетом
срок предоставления рецензий на статьи
20 января 2020г. срок принятия решения о публикации присланных материалов и
рассылки приглашений для участия в конференции и сообщение о
включении статьи в Сборник научных трудов
10 февраля 2020г. размещение на сайте конференции http://conf.ostis.net проекта
программы конференции
15 февраля 2020г. размещение на сайте сборника http://proc.ostis.net Сборника научных
трудов и Программы конференции OSTIS-2020
20 февраля 2020г. регистрация участников и открытие конференции OSTIS-2020
20-22 февраля
2020г.
работа конференции OSTIS-2020
26 февраля 2020г. публикация фоторепортажа и отчёта о проведённой конференции на
сайте конференции: http://conf.ostis.net
15 марта 2020г. загрузка материалов сборника конференции в РИНЦ
ФОРМИРОВАНИЕ ПРОГРАММЫ КОНФЕРЕНЦИИ
Программа конференции формируется Программным комитетом по результатам
рецензирования, представленных статей, а также на основании подтверждения автора(-ов)
статьи о прибытии на конференцию.
КОНТАКТНЫЕ ДАННЫЕ ОРГАНИЗАТОРОВ КОНФЕРЕНЦИИ OSTIS
Вся необходимая информация по предстоящей и предыдущих конференциях OSTIS
находится на сайте конференции http://conf.ostis.net, а также на сайте материалов конференции
http://proc.ostis.net.
По вопросам участия в конференции и решения спорных вопросов обращайтесь:
Методическая и консультативная помощь участникам конференции осуществляется
только через электронную почту конференции.
Конференция проходит в Республике Беларусь, г. Минск, ул. Платонова, 39 (5-ый
учебный корпус Белорусского государственного университета информатики и
радиоэлектроники).
Related Documents
