研究者詳細 - 髙梨克也

A conversational dialogue manager for the humanoid Robot ERICA

移行適格場の情報を考慮したターンテイキング予測

原康平, 井上昂治, 高梨克也, 河原達也

人工知能学会言語・音声理解と対話処理研究会資料 85th 57 - 62 2019年

Pierrick Milhorat, Divesh Lala, Koji Inoue, Tianyu Zhao, Masanari Ishida, Katsuya Takanashi, Shizuka Nakamura, Tatsuya Kawahara

Lecture Notes in Electrical Engineering 510 119 - 131 2019年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：SPRINGER INTERNATIONAL PUBLISHING AG

We present a dialogue system for a conversational robot, Erica. Our goal is for Erica to engage in more human-like conversation, rather than being a simple question-answering robot. Our dialogue manager integrates question-answering with a statement response component which generates dialogue by asking about focused words detected in the user’s utterance, and a proactive initiator which generates dialogue based on events detected by Erica. We evaluate the statement response component and find that it produces coherent responses to a majority of user utterances taken from a human-machine dialogue corpus. An initial study with real users also shows that it reduces the number of fallback utterances by half. Our system is beneficial for producing mixed-initiative conversation.

DOI： 10.1007/978-3-319-92108-2_14

Generating fillers based on dialog act pairs for smooth turn-taking by humanoid robot

その他リンク： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2017.html#MilhoratLIZITNK17

Ryosuke Nakanishi, Koji Inoue, Shizuka Nakamura, Katsuya Takanashi, Tatsuya Kawahara

Lecture Notes in Electrical Engineering 579 91 - 101 2019年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Springer

In spoken dialog systems for humanoid robots, smooth turn-taking function is one of the most important factors to realize natural interaction with users. Speech collisions often occur when a user and the dialog system speak simultaneously. This study presents a method to generate fillers at the beginning of the system utterances to indicate an intention of turn-taking or turn-holding just like human conversations. To this end, we analyzed the relationship between a dialog context and fillers observed in a human-robot interaction corpus, where a user talks with a humanoid robot remotely operated by a human. At first, we annotated dialog act tags in the dialog corpus and analyzed the typical type of a sequential pair of dialog acts, called a DA pair. It is found that the typical filler forms and their occurrence patterns are different according to the DA pairs. Then, we build a machine learning model to predict occurrence of fillers and its appropriate form from linguistic and prosodic features extracted from the preceding and the following utterances. The experimental results show that the effective feature set also depends on the type of DA pair.

DOI： 10.1007/978-981-13-9443-0_8

Turn-taking prediction based on detection of transition relevance place

その他リンク： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2018.html#NakanishiINTK18

Kohei Hara, Koji Inoue, Katsuya Takanashi, Tatsuya Kawahara

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2019-September 4170 - 4174 2019年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：ISCA

We address turn-taking prediction in which spoken dialogue systems predict when to take the conversational floor. In natural conversations, many turn-taking decisions are arbitrary and subjective. In this study, we propose taking into account the concept of the transition relevance place (TRP) for turn-taking prediction. TRP is defined as a timing when the current speaking turn can be completed and other participants are able to take the turn. We conducted annotation of TRP on a human-robot dialogue corpus, ensuring the objectivity of this annotation among annotators. The proposed turn-taking prediction model adopts a two-step approach that detects TRP at first and then predicts a turn-taking event if TRP is detected. Experimental evaluations demonstrate that the proposed model improves the accuracy of turn-taking prediction by incorporating TRP detection.

DOI： 10.21437/Interspeech.2019-1537

Latent character model for engagement recognition based on multimodal behaviors

その他リンク： https://dblp.uni-trier.de/db/conf/interspeech/interspeech2019.html#HaraITK19

Koji Inoue, Divesh Lala, Katsuya Takanashi, Tatsuya Kawahara

Lecture Notes in Electrical Engineering 579 119 - 130 2019年

掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：Springer

Engagement represents how much a user is interested in and willing to continue the current dialogue and is the important cue for spoken dialogue systems to adapt the user state. We address engagement recognition based on listener’s multimodal behaviors such as backchannels, laughing, head nodding, and eye gaze. When the ground-truth labels are given by multiple annotators, they differ according to each annotator due to the different perspectives on the multimodal behaviors. We assume that each annotator has a latent character that affects its perception of engagement. We propose a hierarchical Bayesian model that estimates both the engagement level and the character of each annotator as latent variables. Furthermore, we incorporate other latent variables to map the input feature into a sub-space. The experimental result shows that the proposed model achieves higher accuracy than other models that do not take into account the character.

DOI： 10.1007/978-981-13-9443-0_11

その他リンク： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2018.html#InoueLTK18

How science, technology and innovation can be placed in broader visions - Public opinions from inclusive public engagement activities 査読

Kei Kano, Mitsuru Kudo, Go Yoshizawa, Eri Mizumachi, Makiko Suga, Naonori Akiya, Kuniyoshi Ebina, Takayuki Goto, Masayuki Itoh, Ayami Joh, Haruhiko Maenami, Toshifumi Minamoto, Mikihiko Mori, Yoshitaka Morimura, Tamaki Motoki, Akie Nakayama, Katsuya Takanashi

Journal of Science Communication 18 ( 3 ) A02 2019年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：SCUOLA INT SUPERIORE STUDI AVANZATI-S I S S A-INT SCH ADVANCED STUDIES

This study investigates how different segments of the public, with varying degrees of interest in S & T, could formulate opinions on a broader vision and the role they think STI should play in Japanese society through 2020 (Tokyo's Olympic and Paralympic year) and toward 2030. We conducted nine inclusive public engagement activities. Results indicated that the broad public opinions did not completely overlap with officials' opinions, a value of "open and appropriate" was mainly found from the unengaged public, and the visions and values based on their opinions could well be incorporated into the official document. Engaging the disinterested in S & T remains an issue.

DOI： 10.22323/2.18030202

対話のふるまいに基づくキャラクタ表現の対話コーパスにおける分析

山本賢太, 井上昂治, 中村静, 高梨克也, 河原達也

人工知能学会言語・音声理解と対話処理研究会資料 85th 43 - 48 2019年

Audio-Visual Conversation Analysis by Smart Posterboard and Humanoid Robot

Tatsuya Kawahara, Koji Inoue, Divesh Lala, Katsuya Takanashi

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings 2018-April 6573 - 6577 2018年9月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

This paper addresses audio-visual signal processing for conversation analysis, which involves multi-modal behavior detection and mental-state recognition. We have investigated prediction of turn-taking by the audience in a poster session from their multi-modal behaviors, and found out that the eye-gaze provides an important cue compared with head nodding and verbal backchannels. This finding has been applied to audio-visual speaker diarization by combining eye-gaze information. We are now investigating engagement recognition in human-robot interaction based on the same scheme. Robust and realtime detection of laughing, backchannels and nodding is realized based on LSTM-CTC. We introduce a latent 'character' model to cope with the subjectivity and variations of engagement annotations. Experimental evaluations demonstrate that (1) the latent character model is effective, (2) automatic behavior detection is robust and does not degrade the engagement recognition accuracy, and (3) the eye-gaze is the most important feature among others.

DOI： 10.1109/ICASSP.2018.8461470

人間型ロボットのキャラクタ表現のための対話の振る舞い制御モデル査読

その他リンク： https://dblp.uni-trier.de/db/conf/icassp/icassp2018.html#KawaharaILT18

山本賢太, 井上昂治, 中村静, 高梨克也, 河原達也

人工知能学会論文誌 33 ( 5 ) C - I37_1 2018年9月

Emotion recognition by combining prosody and sentiment analysis for expressing reactive emotion by humanoid robot

Yuanchao Li, Carlos Toshinori Ishi, Nigel Ward, Koji Inoue, Shizuka Nakamura, Katsuya Takanashi, Tatsuya Kawahara

Proceedings - 9th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017 2018-February 1356 - 1359 2018年2月

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：IEEE

In order to achieve rapport in human-robot interaction, it is important to express a reactive emotion that matches with the user's mental state. This paper addresses an emotion recognition method which combines prosody and sentiment analysis for the system to properly express reactive emotion. In the user emotion recognition module, valence estimation from prosodic features is combined with sentiment analysis of text information. Combining the two information sources significantly improved the valence estimation accuracy. In the reactive emotion expression module, the system's emotion category and level are predicted using the parameters estimated in the recognition module, based on distributions inferred from human-human dialog data. Subjective evaluation results show that the proposed method is effective for expressing human-like reactive emotion.

DOI： 10.1109/APSIPA.2017.8282243

潜在キャラクタモデルによる聞き手のふるまいに基づく対話エンゲージメントの推定査読

その他リンク： https://dblp.uni-trier.de/db/conf/apsipa/apsipa2017.html#LiIWINTK17

瞬発的運動のジレンマ─雪上での木遣りの事例分析から─

高梨克也

生態心理学研究 11 ( 2 ) 30 - 33 2018年

井上昂治, Divesh Lala, 吉井和佳, 高梨克也, 河原達也

人工知能学会論文誌 33 ( 1 ) DSH - F_1-12 2018年

記述言語：日本語掲載種別：研究論文（学術雑誌）出版者・発行元：一般社団法人人工知能学会

<p>This article addresses the estimation of engagement level based on the listener's behaviors such as backchannel, laughing, head nodding, and eye-gaze. Engagement is defined as the level of how much a user is being interested in and willing to continue the current interaction. When the engagement level is evaluated by multiple annotators, the criteria for annotating the engagement level would depend on each annotator. We assume that each annotator has its own character which affects the way of perceiving the engagement level. We propose a latent character model which estimates the engagement level and also the character of each annotator as a latent variable. The experimental results show that the latent character model can predict the engagement label of each annotator in higher accuracy than other models which do not take the character into account.</p>

DOI： 10.1527/tjsai.DSH-F

J-GLOBAL

自律型アンドロイドのキャラクタ表現のための対話の振る舞い制御モデルの構築と評価

山本賢太, 井上昂治, 中村静, 高梨克也, 河原達也

情報処理学会全国大会講演論文集 80th ( 2 ) 407 - 408 2018年

記述言語：日本語

本研究では,音声対話を通して自律型アンドロイドが与えられた社会的役割に適したキャラクタ表現方法を提案する.これまでに,キャラクタを構成する外向性,情緒不安定性,丁寧さという3要素と音声特徴量との関係性を調査した.印象評定の統計分析により,キャラクタ印象に影響する4つの音声特徴量が確認された.この結果を用いて,アンドロイドに与えられたキャラクタに合わせて発話量,相槌の頻度,フィラーの頻度,交替潜時の長さという4つの特徴量を制御するモデルを構築した.印象評価実験を行い,本モデルを用いることで,外向性と丁寧さの要素ついてはキャラクタ表現の可能性が確認された.

CiNii Books

J-GLOBAL

相槌・フィラー予測とのマルチタスク学習による円滑なターンテイキング

自律型アンドロイドERICAにおけるエンゲージメント推定に基づく音声対話システム

井上昂治, LALA Divesh, 高梨克也, 河原達也

日本音響学会研究発表会講演論文集(CD-ROM) 2018 2018年

原康平, 井上昂治, 高梨克也, 河原達也

情報処理学会全国大会講演論文集 80th ( 2 ) 409 - 410 2018年

記述言語：日本語

人との自然な対話を指向したヒューマンロボットインタラクションにおいて,円滑なターンテイキングの実現は重要な課題である.円滑なターンテイキングを実現するためには,発話末における話者交替および継続の予測が必要である.また,これに関連するふるまいとして,相槌およびフィラーの生成が挙げられる.従来研究では,これらを予測する課題は独立に扱われてきた.本研究では,マルチタスク学習の枠組みにより,これらのふるまいの予測を同一のモデルで扱うことで,ターンテイキング予測の精度向上を目指す.自律型アンドロイドERICAを用いた対話コーパスでの実験により,提案手法の予測精度がシングルタスクの場合よりも向上することを示す.

CiNii Books

J-GLOBAL

相槌・フィラー予測とのマルチタスク学習によるターンテイキング予測

原康平, 井上昂治, 高梨克也, 河原達也

人工知能学会言語・音声理解と対話処理研究会資料 84th 1 - 6 2018年

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Koji Inoue, Divesh Lala, Katsuya Takanashi, Tatsuya Kawahara

APSIPA Transactions on Signal and Information Processing 7 2018年

記述言語：英語掲載種別：研究論文（学術雑誌）出版者・発行元：CAMBRIDGE UNIV PRESS

Engagement represents how much a user is interested in and willing to continue the current dialogue. Engagement recognition will provide an important clue for dialogue systems to generate adaptive behaviors for the user. This paper addresses engagement recognition based on multimodal listener behaviors of backchannels, laughing, head nodding, and eye gaze. In the annotation of engagement, the ground-truth data often differs from one annotator to another due to the subjectivity of the perception of engagement. To deal with this, we assume that each annotator has a latent character that affects his/her perception of engagement. We propose a hierarchical Bayesian model that estimates both engagement and the character of each annotator as latent variables. Furthermore, we integrate the engagement recognition model with automatic detection of the listener behaviors to realize online engagement recognition. Experimental results show that the proposed model improves recognition accuracy compared with other methods which do not consider the character such as majority voting. We also achieve online engagement recognition without degrading accuracy.

DOI： 10.1017/ATSIP.2018.11

Prediction of turn-taking using multitask learning with prediction of backchannels and fillers

Kohei Hara, Koji Inoue, Katsuya Takanashi, Tatsuya Kawahara

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2018-September 991 - 995 2018年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：ISCA-INT SPEECH COMMUNICATION ASSOC

We address prediction of turn-taking considering related behaviors such as backchannels and fillers. Backchannels are used by the listeners to acknowledge that the current speaker can hold the turn. On the other hand, fillers are used by the prospective speakers to indicate a will to take a turn. We propose a turn-taking model based on multitask learning in conjunction with prediction of backchannels and fillers. The multitask learning of LSTM neural networks shared by these tasks allows for efficient and generalized learning, and thus improves prediction accuracy. Evaluations with two kinds of dialogue corpora of human-robot interaction demonstrate that the proposed multitask learning scheme outperforms the conventional single-task learning.

DOI： 10.21437/Interspeech.2018-1442

Engagement recognition in spoken dialogue via neural network by aggregating different annotators' models

その他リンク： https://dblp.uni-trier.de/db/conf/interspeech/interspeech2018.html#HaraITK18

Koji Inoue, Divesh Lala, Katsuya Takanashi, Tatsuya Kawahara

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2018-September 616 - 620 2018年

記述言語：英語掲載種別：研究論文（国際会議プロシーディングス）出版者・発行元：ISCA-INT SPEECH COMMUNICATION ASSOC

This paper addresses engagement recognition based on four multimodal listener behaviors - backchannels, laughing, eye-gaze, and head nodding. Engagement is an indicator of how much a user is interested in the current dialogue. Multiple third-party annotators give ground truth labels of engagement in a human-robot interaction corpus. Since perception of engagement is subjective, the annotations are sometimes different between individual annotators. Conventional methods directly use integrated labels, such as those generated through simple majority voting, and do not consider each annotator's recognition. We propose a two-step engagement recognition where each annotator's recognition is modeled and the different annotators' models are aggregated to recognize the integrated label. The proposed neural network consists of two parts. The first part corresponds to each annotator's model which is trained with the corresponding labels independently. The second part aggregates the different annotators' models to obtain one integrated label. After each part is pre-trained, the whole network is fine-tuned through back-propagation of prediction errors. Experimental results show that the proposed network outperforms baseline models which directly recognize the integrated label without considering differing annotations.

DOI： 10.21437/Interspeech.2018-2067