At present, artificial intelligent (AI) technology has made great achievements in various fields including computer
vision, automatic driving, speech recognition, natural language processing, etc.
On the other hand, the convolutional neural network (CNN), especially the deep neural network, which
is one of the fundamental technologies in the field of AI,
demands an overwhelming amount of computations and memories. At the same time, massive redundant computations are
caused by excessive parameters, As a result, limitations such as hardware resources, memory shortage, power consumption,
and time-consuming largely restrict the applications of AI.
This research aims to realize the high-performance AI from both software-level and hardware-level.
1. More Compact, More Smarter AI Models
The study aims to automatically optimize the architecture of the neural network by selecting appropriate learning rate,
adjusting layers as well as channels, preserving important connection nodes and cutting off
redundant neurons in the training phase, and so on. Thus, the redundant computation of the model is automatically swept
and the optimization of the model is automatically performed. It makes the AI more compact and smarter,
The figure shows the flow of the study.
2. Acceleration of AI Models by FPGA
FPGA provides promising solutions to accelerate neural networks at the hardware level. There are mainly two ways
to deploy AI models on FPGA: the HLS and the Vitis AI. HLS
is a high-level synthesis tool to accelerate design implementation by enabling C, C++, and System C specifications
to be directly targeted into FPGAs without the need to manually create RTL. The Vitis AI development environment is a
convenient and efficient tool for deploying and accelerating AI inference on Xilinx hardware platforms
including both Edge devices and Alevo accelerator cards. The platform makes it possible for developers
even without FPGA knowledge to implement deep-learning model applications on FPGA platform. This
section makes an overview of the Vitis AI in terms of the process of implementing the AI inference on FPGA.
In this research, we aim to accelerate AI models by utilizing FPGA.
The following video shows the process from the optimization to the deployment of a AI model on FPGA.
3. Processor Architecture
The processor architecture is the basic technology of computer science.
Further improvement of speed is necessary for the evolution of technology and the scope of application.
We have attempted to speed up the processor at the architecture level.
Data dependence and bifurcation dependence are proposed for data dependence and control dependence.
Further, several proposals have been made to optimize the branch prediction miss penalty.
Although computer architecture is a traditional infrastructure technology,
it is difficult to understand for the reason that it is not widely used as software technology.
There are many cases that are not valued. However, better architectures are needed to design better systems.
The diagram shows the architecture of our proposal.
There have been many uncensored ancient documents, and decoding is quite difficult due to the deterioration.
Many potential knowledges are hidden in these ancient literature. In this study,
the ancient literature of China and the ancient literature of India are decoded using image processing,
deep learning, and big data analysis. And the reorganization of a great quantity Asian ancient culture has been implemented.
The research consists of the following sub-themes.
1. DL-based Recognition of Oracle Bone Inscription
Oracle Bone Inscription is the hieroglyphics of the Yin Dynasty which is more than 3000 years ago,
and it is said to be the originator of kanji. The decoding of the Oracle characters is very
important for the study of the origin and history of characters, but there is a problem that
it is difficult to recognize them due to the deterioration. We aim at the recognition of
the oracle using image processing and deep learning. In the following figures, we extract the
segment features of the oracle using the Hough transform and calculate the similarity of the
line segment features between the callus and the template by deep learning including CNN and SSD techniques.
The CNN is a method of recognizing each character by self-learning from identified characters.
In SSD, segmentation and classification of characters are performed simultaneously.
There are both shortcomings and advantages in the two methods, it is expected to improve the technique with your power!
2. Recognition and Potential Knowledge Extraction of Characters using DL and Big Data Analysis
The interpretation, analysis, and knowledge extraction of ancient literature will
help in the study of history, climate change, and prediction of natural disasters.
In the absence of paper, people engrave sentences on stones, metals, etc.,
and record important events, incidents, people, etc. The custom had been continued
for a long time. The book was made by putting paper on them and copying it with ink.
In short, the book is important ancient literature with a long history. However,
there are many cases in which a variety of characters are used depending on the age,
and they are hard to be recognized. In addition, research on potential knowledge extraction
has not been advanced from the present document. The purpose of this research is to make
a time‐space database of keywords of the book and to discover potential knowledge by
recognizing the characters of the book.
The following figure is an example of character recognition using deep learning and
big data analysis technique. The left unknown book shows that it can be recognized correctly by the method.
3. DL-based Recognition of Kuzushiji Characters
Japanese classical books record much information, and the deciphering of these classical books
is very useful for the study of history, politics and culture. However, there are many Japanese classical
documents that have not been deciphered. In recent years, with the rapid development of artificial intelligence technology,
researchers attempt to decipher the information recorded in Japanese classical literature through deep learning.
However, there are many cases where Japanese classical
books are recorded with Kuzushiji, and there are many variations for these characters, so it is very
difficult to recognize these characters. In addition, because the characters are connected among each other generally,
the difficulty of automatic decoding of classical documents is increasing.
This research aims at automatic recognition of Japanese classical literature using multiple deep
learning models. The following figure is one of the proposed methods.
First of all, the line is extracted using the Aru net. Then, the character size is estimated
for the line of the extracted text and stripped and the character candidate is cut.
Lastly, LeNet is used as a recognition result from a character candidate.
4. Restoration of Ancient Literature using Generative Adversarial Networks (Gan)
We have been aiming at automatic recognition of old characters in order to decipher the Japanese Kuzushiji,
the script, and the Japanese classics using AI technology. However, the difficulty of automatic
recognition is increasing due to the occurrence of character defects caused by insect eating, deterioration, stain, etc.
This research aims at the restoration of character deficiency of cultural property such as classical documents
using AI technology. In the study, we apply Gan technique to repair
the Japanese classical literature which has been deteriorated by the insect eating, deterioration, and dirt.
The research contributes to the reorganization and protection of cultural heritage.
1.AI-based Facial Expression Recognition IoT System
Due to the super-strength work, physical condition management, especially fatigue and mental health management,
is important for modern people. This research aims to realize a facial expression recognition
IoT system using deep learning and to detect the fatigue and mental problems at an early stage based on facial
expression recognition. By this, we aim to contribute to the realization of a safe, secure and comfortable society.
Specifically, on the edge side, a person is detected and and recognized by a motion sensor, then the face is extracted.
The recognized person information and facial image are transferred to the cloud (server), and facial
expressions are identified by deep learning. By using the accumulated facial expressions, the changes in facial
expressions are quantitatively evaluated to predict the fatigue and mental conditions.
Facial expression recognition IoT system configuration diagram using deep learning.
IoT system configuration diagram of facial expression recognition using deep learning.
2.Recognize Anywhere, Anytime (Ancient Literature)）
The Japanese classical books record a lot of information, and the decoding of these classical
books is very useful for the study of history, politics, and culture. Recognizing anywhere,
anytime is an important technique that makes it convenient to organize. Also, We think it can be
of interest to many people.
The goal of this research is to be able to recognize ancient literature anytime,
anywhere by utilizing IoT technology.
First, a recognition model for deep
learning is created on the server. Then, the document image to be recognized is transferred to
the server via the Web for recognition. We also created an adroit application and used
an android application on a mobile device such as a tablet. It consists of a web-based
recognition system and an android-based recognition system.
1. AI for Tidying Up Tableware
In Japan, where the working population is declining, it is an important issue
to replace the working population by using robots and AI. This research aims
at automatic tableware collection by a mobile manipulator.
In this research, a camera is attached to the tip of a mobile robot to take
photographs. Tableware is recognized using AI, and the collection
of tableware is automatically realized. Finally, the tableware is transported to the kitchen.
Video of Tidying Up Tableware(Youtube)
2. AI-aided Medical Diagnosis System
The study aims to assist the medical diagnosis using AI technology. Currently, the research
contributes to the diagnosis of Apathy in the elderly by using AI technology.
Early detection for apathy is very important because Apathy can also occur in depression and
Parkinson's disease along with a state of emotionlessness. In this study, Doppler Radar
data is adopted and preprocessed, and then normal/abnormal discrimination is
performed by machine learning. The figure below shows the flow of the research.
3. Social Distance and Mask Wearing Check System for Covid-19
We aim to realize a safe and secure society by sensitively discovering and responding
promptly to the demands of the times and industry. Here, we have been building a mask
and social distance checking system for COVID-19. Currently, we are studying for practical use.
4. Prediction of Metal Deterioration
Fatigue crack growth detection is very important for avoiding industrial risks.
The research aims to predict metal deterioration using machine learning.
