Authorship：Lead author  

researchmap

Language：Japanese   Publisher：Japan Association for Medical Informatics  

Practical design of ICT systems and workflow of real time telemedicine requires understanding of communication processes in telemedicine. This study reports the first trial of communication analysis in a simulated tele-auscultation experiment as the primary study for communication analysis in telemedicine. In the experiment, a remote doctor instructs a helper with no auscultation skills to put a chest piece of a stethoscope on a target point by using a vision-based tele-navigation system. An auscultation trainer is used as a client to simulate lung diseases. Communication between the remote doctor and the helper is analyzed with survey of questionnaire and interview to the subjects, and observation of multi angle videos such as the views in the tele-auscultation support system and the external appearance of the experiment. The results show basic knowledge of tele-auscultation communication analysis such as multimodal and multichannel interactional patterns of remote stethoscope navigation.

DOI： 10.14948/jami.35.297

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publisher：Japanese Cognitive Science Society  

This study examines the properties of gestures that serve to convey an image of the<br>speaker. The gesture observed can be perceived as a sub-type of gestures that linguistic<br>anthropologist Charles Goodwin termed "environmentally coupled gestures" in that the<br> meaning of the gesture is determined not by its semiotic property but by the mutual<br>elaboration of the gesture and other semiotic resources such as artifacts, utterance, and<br>gaze. A conversation analysis was conducted based on data collected during interaction<br>among members of a project team constructing an installation for a science museum.<br>The results show that team members recurrently re-do gestures that convey the image<br>of the speaker along with another gesture constructed to be coupled with the original<br>gesture.

DOI： 10.11225/jcss.22.557

CiNii Books

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：Japanese  

DOI： 10.14943/56173

CiNii Books

researchmap

Other Link： http://hdl.handle.net/2115/49443

Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

Based on ethnography in a scientific museum, this article analyzes a single case of successive group problem-solving interactions for designing an exhibition as a future space. In the process of multimodal interaction in multiple spaces, heterogeneous resources in real spaces became representations of objects not yet existing and were continuously transformed with reference to a framework for collaborative problem-solving. © 2012 Springer-Verlag.

DOI： 10.1007/978-3-642-32090-3_25

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/jsai/jsai2011.html#TakanashiH11

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：English  

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.23.4_538

CiNii Books

J-GLOBAL

researchmap

Other Link： http://id.nii.ac.jp/1004/00007026/

Language：English   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER-VERLAG BERLIN  

Recent interests in conversation in the field of artificial intelligence have expanded beyond the development of particular task-oriented dialogue systems toward technologies for supporting human-human communication in various circumstances. Within such communication supportive approaches, the importance of the analysis of multi-party conversation has increasingly been recognized. In accordance with these orientations, this article outlines a three-party conversation corpus built by the National Institute of Information and Communications Technology, and introduces three preliminary analyses of it that will contribute to the development of Conversational Informatics: The characteriscits of turn-taking procedure in three-person conversation; assessment sequential patterns that appeared in the data; and shared knowledge and interpersonal relationships between participants observable from the assessment sequences in triadic conversation. © Springer-Verlag Berlin Heidelberg 2006.

DOI： 10.1007/11780496_41

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/jsai/jsai2005.html#TakanashiFKTI05

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.16.6_799

CiNii Books

J-GLOBAL

researchmap

Other Link： http://id.nii.ac.jp/1004/00005348/

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese  

CiNii Books

researchmap

Language：Japanese   Publisher：日本語用論学会  

CiNii Books

researchmap

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

DOI： 10.24525/shitsuforum.14.0_91

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

DOI： 10.24525/shitsuforum.13.0_37

researchmap

DOI： 10.24525/shitsuforum.13.0_5

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

DOI： https://doi.org/10.1527/tjsai.36-5_H-L51

Publishing type：Research paper (scientific journal)   Publisher：Japanese Society for Artificial Intelligence  

DOI： 10.1527/tjsai.36-5_h-l51

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Transactions of the Japanese Society for Artificial Intelligence  

An attentive listening system for autonomous android ERICA is presented. Our goal is to realize a human-like natural attentive listener for elderly people. The proposed system generates listener responses: backchannels, repeats, elaborating questions, assessments, and generic responses. The system incorporates speech processing using a microphone array and real-time dialogue processing including continuous backchannel prediction and turn-taking prediction. In this study, we conducted a dialogue experiment with elderly people. The system was compared with a WOZ system where a human operator played the listener role behind the robot. As a result, the system showed comparable scores in basic skills of attentive listening, such as easy to talk, seriously listening, focused on the talk, and actively listening. It was also found that there is still a gap between the system and the human (WOZ) for high-level attentive listening skills such as dialogue understanding, showing interest, and empathy towards the user.

DOI： 10.1527/tjsai.36-5_H-L51

Scopus

Other Link： https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85114196962&origin=inward

Publishing type：Research paper (international conference proceedings)  

We address character (personality) expression for a spoken dialogue system in order to accommodate it in particular dialogue tasks and social roles. While conventional studies investigated controlling the linguistic expressions, we focus on spoken dialogue behaviors to express systems’ characters. Specifically, we investigate spoken dialogue behaviors such as utterance amount, backchannel frequency, filler frequency, and switching pause length in order to express three character traits: extroversion, emotional instability, and politeness. In this study, we evaluate this model with a natural spoken dialogue corpus. The results reveal that this model expresses reasonable characters according to the dialogue tasks and the participant roles. Furthermore, it is also shown that this model is able to express different characters among participants given the same role. A subjective experiment demonstrated that subjects could perceive the characters expressed by the model.

DOI： 10.1007/978-981-15-8395-7_1

Scopus

researchmap

Publishing type：Part of collection (book)   Publisher：Springer  

In order to make conversational agents or robots conduct human-like behaviors, it is important to design a model of the system internal states. In this paper, we address a model of favorable impression to the dialogue partner. The favorable impression is modeled to change according to user’s dialogue behaviors and also affect following dialogue behaviors of the system, specifically selection of utterance constructional units. For this modeling, we propose a hierarchical structure of logistic regression models. First, from the user’s dialogue behaviors, the model estimates the level of user’s favorable impression to the system and also the level of the user’s interest in the current topic. Then, based on the above results, the model predicts the system’s favorable impression to the user. Finally, the model determines selection of utterance constructional units in the next system turn. We train each of the logistic regression models individually with a small amount of annotated data of favorable impression. Afterward, the entire multi-layer network is fine-tuned with a larger amount of dialogue behavior data. An experimental result shows that the proposed method achieves higher accuracy on the selection of the utterance constructional units, compared with methods that do not take into account the system internal states.

DOI： 10.1007/978-981-15-9323-9_2

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2019.html#TanakaINTK19

Publishing type：Part of collection (book)   Publisher：Springer  

We demonstrate a job interview dialogue with the autonomous android ERICA which plays the role of an interviewer. Conventional job interview dialogue systems ask only pre-defined questions. The job interview system of ERICA generates follow-up questions based on the interviewee’s response on the fly. The follow-up questions consist of two kinds of approaches: selection-based and keyword-based. The first type question is based on selection from a pre-defined question set, which can be used in many cases. The second type of question is based on a keyword extracted from the interviewee’s response, which digs into the interviewee’s response dynamically. These follow-up questions contribute to realizing natural and trained dialogue.

DOI： 10.1007/978-981-15-9323-9_25

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2019.html#InoueHLNTK19

Publishing type：Part of collection (book)   Publisher：Springer  

We address an application of engagement recognition in human-robot dialogue. Engagement is defined as how much a user is interested in the current dialogue, and keeping users engaged is important for spoken dialogue systems. In this study, we apply a real-time engagement recognition model to laboratory guide by autonomous android ERICA which plays the role of the guide. According to an engagement score of a user, ERICA generates adaptive behaviors consisting of feedback utterances and additional explanations. A subject experiment showed that the adaptive behaviors increased both the engagement score and related subjective scores such as interest and empathy.

DOI： 10.1007/978-981-15-9323-9_11

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2019.html#InoueLYTK19

Publishing type：Research paper (international conference proceedings)  

Example-based dialogue systems are often used in practice because of their robustness and simple architecture. However, when these systems are given out-of-database questions that are not registered in the question-response database, they have to respond with a fixed backup response, which can make users disengaged in the dialogue. In this study, we address response generation for out-of-database questions to make users perceive that the system understands the question itself. We define question types observed in the speed-dating scenario which is based on open-domain dialogue. Then we define possible response frames for each question type. We propose a sequence-to-sequence model that directly generates an appropriate response frame from an input question sentence in an end-to-end manner. The proposed model also explicitly integrates a question type classification to take into account the question type of the out-of-database question. Experimental results show that integrating the question type classification improved the response generation, and could exactly match 69.2% of response frames provided by human annotators.

DOI： 10.1007/978-981-15-8395-7_23

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)   Participation form：Joint(The vice charge)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

A job interview is a domain that takes advantage of an android robot's human-like appearance and behaviors. In this work, our goal is to implement a system in which an android plays the role of an interviewer so that users may practice for a real job interview. Our proposed system generates elaborate follow-up questions based on responses from the interviewee. We conducted an interactive experiment to compare the proposed system against a baseline system that asked only fixed-form questions. We found that this system was significantly better than the baseline system with respect to the impression of the interview and the quality of the questions, and that the presence of the android interviewer was enhanced by the follow-up questions. We also found a similar result when using a virtual agent interviewer, except that presence was not enhanced.

DOI： 10.1145/3382507.3418839

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icmi/icmi2020.html#InoueHLYNTK20

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Publishing type：Research paper (scientific journal)   Publisher：Japanese Society for Artificial Intelligence  

DOI： 10.1527/tjsai.35-5_d-k43

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese  

J-GLOBAL

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computational Linguistics  

researchmap

Other Link： https://dblp.uni-trier.de/conf/sigdial/2020

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

This paper addresses character expression for humanoid robots that play a social role via spoken dialogue so that the character matches to the given social role such as a lab guide or a counselor. While conventional methods of character expression mostly focused on changing the style of utterance texts, this study focuses on dialogue behavior features that may affect the impression of spoken dialogue. Specifically, we use five dialogue behavior features: utterance amount, backchannel frequency, backchannel variety, filler frequency, and switching pause length (the time until the system responds). We adopt three character traits of extroversion, emotional instability, and politeness for character expression. We then investigate the relationship between the dialogue behavior features and the character traits by conducting subjective evaluations. A statistical analysis of the subjective evaluations shows that the dialogue behavior features except for the backchannel variety are related to either of the character traits. By using the subjective evaluation scores on the relevant traits, we can train models to control the dialogue behavior features of a robot according to the desired character. Another experimental evaluation demonstrates the feasibility of character expression with regard to the traits of extroversion and politeness.

DOI： 10.23919/APSIPA.2018.8659624

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/apsipa/apsipa2018.html#YamamotoINTK18

Language：Japanese  

音声対話システムが円滑な対話を実現するためには，次に話者が交替するか否か（ターンテイキング）の予測が重要である．本研究では，ターンテイキングの予測において，移行適格場 （TRP）の情報を考慮する．TRPとは，話者交替が起こりうる箇所を指し，ターンテイキングのふるまいは，TRPか否かに依存すると考えらえる． TRPは客観的に定義できるため，予測が比較的容易である．提案モデルは，ニューラルネットに基づく二段階の予測を行う．はじめに，TRPの予測を行う．次に，TRP予測の結果を用いてターンテイキングの予測を行う．評価実験により，TRPの情報を用いることで，ターンテイキングの予測精度が向上することを確認した．

CiNii Books

researchmap

Language：Japanese  

対話システムが人間らしい対話を実現するためには，システム自身の趣味や嗜好といったプロファイル情報を設定することが有効といえる．プロファイルの一貫性を保つためには，想定質問とその応答の用例を用意しておく必要がある．しかし，用例がない質問がなされた場合，従来のシステムでは「お答えできません」などの画一的な応答を返してしまう．そこで，これらの質問を「個人情報」，「好み」，「習慣」などのタイプに分類し，それぞれに対応した応答を生成する．これにより，その質問をシステムが理解しているかのようにみせかける．実験では，自律型アンドロイドERICAを用いたお見合い対話コーパスにおいて提案システムの有効性を検証する．

CiNii Books

researchmap

Language：Japanese  

我々は自律型アンドロイドERICAの研究開発を進めており，特定の社会的役割が与えられた状況での対話を扱っている．この際に，その社会的役割に適したキャラクタ（性格）をアンドロイドが表出することで，より自然な対話が実現できると考えらえる．これまでに，発話量，相槌，フィラー，交替潜時といった対話におけるシステムのふるまいを制御することで，外向性，情緒不安定性，丁寧さから成るキャラクタを表現するモデルを提案した．本稿では，遠隔操作されたアンドロイドとの対話コーパスを用いて，実際の対話のふるまいと知覚されるキャラクタとの関係が，提案モデルでも実現できることを検証した．

CiNii Books

researchmap

Language：Japanese  

音声対話システムに人間らしい対話を実現させるためには，システムが内部状態を持ち，それに基づきふるまいを動的に変化させることが重要と考えられる．本研究では，内部状態として，システムがユーザに対して抱く好感を扱う．システムの好感を表現するために，ここではシステム発話の構成要素を変化させる．提案モデルは二段階のニューラルネットワークで構成される．はじめに，ユーザの話し方や聞き方のふるまいに基づきシステムがユーザに抱く好感を生成する．次に，好感と上記のふるまいに基づきシステム発話の構成要素を選択する．また，上記を自律型アンドロイドERICAのお見合い対話システムに実装した．

CiNii Books

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER INTERNATIONAL PUBLISHING AG  

We present a dialogue system for a conversational robot, Erica. Our goal is for Erica to engage in more human-like conversation, rather than being a simple question-answering robot. Our dialogue manager integrates question-answering with a statement response component which generates dialogue by asking about focused words detected in the user’s utterance, and a proactive initiator which generates dialogue based on events detected by Erica. We evaluate the statement response component and find that it produces coherent responses to a majority of user utterances taken from a human-machine dialogue corpus. An initial study with real users also shows that it reduces the number of fallback utterances by half. Our system is beneficial for producing mixed-initiative conversation.

DOI： 10.1007/978-3-319-92108-2_14

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2017.html#MilhoratLIZITNK17

Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

In spoken dialog systems for humanoid robots, smooth turn-taking function is one of the most important factors to realize natural interaction with users. Speech collisions often occur when a user and the dialog system speak simultaneously. This study presents a method to generate fillers at the beginning of the system utterances to indicate an intention of turn-taking or turn-holding just like human conversations. To this end, we analyzed the relationship between a dialog context and fillers observed in a human-robot interaction corpus, where a user talks with a humanoid robot remotely operated by a human. At first, we annotated dialog act tags in the dialog corpus and analyzed the typical type of a sequential pair of dialog acts, called a DA pair. It is found that the typical filler forms and their occurrence patterns are different according to the DA pairs. Then, we build a machine learning model to predict occurrence of fillers and its appropriate form from linguistic and prosodic features extracted from the preceding and the following utterances. The experimental results show that the effective feature set also depends on the type of DA pair.

DOI： 10.1007/978-981-13-9443-0_8

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2018.html#NakanishiINTK18

Publishing type：Research paper (international conference proceedings)   Publisher：ISCA  

We address turn-taking prediction in which spoken dialogue systems predict when to take the conversational floor. In natural conversations, many turn-taking decisions are arbitrary and subjective. In this study, we propose taking into account the concept of the transition relevance place (TRP) for turn-taking prediction. TRP is defined as a timing when the current speaking turn can be completed and other participants are able to take the turn. We conducted annotation of TRP on a human-robot dialogue corpus, ensuring the objectivity of this annotation among annotators. The proposed turn-taking prediction model adopts a two-step approach that detects TRP at first and then predicts a turn-taking event if TRP is detected. Experimental evaluations demonstrate that the proposed model improves the accuracy of turn-taking prediction by incorporating TRP detection.

DOI： 10.21437/Interspeech.2019-1537

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/interspeech/interspeech2019.html#HaraITK19

Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

Engagement represents how much a user is interested in and willing to continue the current dialogue and is the important cue for spoken dialogue systems to adapt the user state. We address engagement recognition based on listener’s multimodal behaviors such as backchannels, laughing, head nodding, and eye gaze. When the ground-truth labels are given by multiple annotators, they differ according to each annotator due to the different perspectives on the multimodal behaviors. We assume that each annotator has a latent character that affects its perception of engagement. We propose a hierarchical Bayesian model that estimates both the engagement level and the character of each annotator as latent variables. Furthermore, we incorporate other latent variables to map the input feature into a sub-space. The experimental result shows that the proposed model achieves higher accuracy than other models that do not take into account the character.

DOI： 10.1007/978-981-13-9443-0_11

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iwsds/iwsds2018.html#InoueLTK18

Language：English   Publishing type：Research paper (scientific journal)   Publisher：SCUOLA INT SUPERIORE STUDI AVANZATI-S I S S A-INT SCH ADVANCED STUDIES  

This study investigates how different segments of the public, with varying degrees of interest in S & T, could formulate opinions on a broader vision and the role they think STI should play in Japanese society through 2020 (Tokyo's Olympic and Paralympic year) and toward 2030. We conducted nine inclusive public engagement activities. Results indicated that the broad public opinions did not completely overlap with officials' opinions, a value of "open and appropriate" was mainly found from the unengaged public, and the visions and values based on their opinions could well be incorporated into the official document. Engaging the disinterested in S & T remains an issue.

DOI： 10.22323/2.18030202

Web of Science

Scopus

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

This paper addresses audio-visual signal processing for conversation analysis, which involves multi-modal behavior detection and mental-state recognition. We have investigated prediction of turn-taking by the audience in a poster session from their multi-modal behaviors, and found out that the eye-gaze provides an important cue compared with head nodding and verbal backchannels. This finding has been applied to audio-visual speaker diarization by combining eye-gaze information. We are now investigating engagement recognition in human-robot interaction based on the same scheme. Robust and realtime detection of laughing, backchannels and nodding is realized based on LSTM-CTC. We introduce a latent 'character' model to cope with the subjectivity and variations of engagement annotations. Experimental evaluations demonstrate that (1) the latent character model is effective, (2) automatic behavior detection is robust and does not degrade the engagement recognition accuracy, and (3) the eye-gaze is the most important feature among others.

DOI： 10.1109/ICASSP.2018.8461470

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icassp/icassp2018.html#KawaharaILT18

Publishing type：Research paper (scientific journal)   Publisher：Japanese Society for Artificial Intelligence  

DOI： 10.1527/tjsai.c-i37

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

In order to achieve rapport in human-robot interaction, it is important to express a reactive emotion that matches with the user's mental state. This paper addresses an emotion recognition method which combines prosody and sentiment analysis for the system to properly express reactive emotion. In the user emotion recognition module, valence estimation from prosodic features is combined with sentiment analysis of text information. Combining the two information sources significantly improved the valence estimation accuracy. In the reactive emotion expression module, the system's emotion category and level are predicted using the parameters estimated in the recognition module, based on distributions inferred from human-human dialog data. Subjective evaluation results show that the proposed method is effective for expressing human-like reactive emotion.

DOI： 10.1109/APSIPA.2017.8282243

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/apsipa/apsipa2017.html#LiIWINTK17

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Publisher：The Japanese Society for Artificial Intelligence  

This article addresses the estimation of engagement level based on the listener’s behaviors such as backchannel, laughing, head nodding, and eye-gaze. Engagement is defined as the level of how much a user is being interested in and willing to continue the current interaction. When the engagement level is evaluated by multiple annotators, the criteria for annotating the engagement level would depend on each annotator. We assume that each annotator has its own character which affects the way of perceiving the engagement level. We propose a latent character model which estimates the engagement level and also the character of each annotator as a latent variable. The experimental results show that the latent character model can predict the engagement label of each annotator in higher accuracy than other models which do not take the character into account.

DOI： 10.1527/tjsai.DSH-F

Scopus

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

This paper addresses engagement recognition based on four multimodal listener behaviors - backchannels, laughing, eye-gaze, and head nodding. Engagement is an indicator of how much a user is interested in the current dialogue. Multiple third-party annotators give ground truth labels of engagement in a human-robot interaction corpus. Since perception of engagement is subjective, the annotations are sometimes different between individual annotators. Conventional methods directly use integrated labels, such as those generated through simple majority voting, and do not consider each annotator's recognition. We propose a two-step engagement recognition where each annotator's recognition is modeled and the different annotators' models are aggregated to recognize the integrated label. The proposed neural network consists of two parts. The first part corresponds to each annotator's model which is trained with the corresponding labels independently. The second part aggregates the different annotators' models to obtain one integrated label. After each part is pre-trained, the whole network is fine-tuned through back-propagation of prediction errors. Experimental results show that the proposed network outperforms baseline models which directly recognize the integrated label without considering differing annotations.

DOI： 10.21437/Interspeech.2018-2067

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/interspeech/interspeech2018.html#InoueLTK18

Language：English   Publishing type：Research paper (scientific journal)   Publisher：CAMBRIDGE UNIV PRESS  

Engagement represents how much a user is interested in and willing to continue the current dialogue. Engagement recognition will provide an important clue for dialogue systems to generate adaptive behaviors for the user. This paper addresses engagement recognition based on multimodal listener behaviors of backchannels, laughing, head nodding, and eye gaze. In the annotation of engagement, the ground-truth data often differs from one annotator to another due to the subjectivity of the perception of engagement. To deal with this, we assume that each annotator has a latent character that affects his/her perception of engagement. We propose a hierarchical Bayesian model that estimates both engagement and the character of each annotator as latent variables. Furthermore, we integrate the engagement recognition model with automatic detection of the listener behaviors to realize online engagement recognition. Experimental results show that the proposed model improves recognition accuracy compared with other methods which do not consider the character such as majority voting. We also achieve online engagement recognition without degrading accuracy.

DOI： 10.1017/ATSIP.2018.11

Web of Science

Scopus

researchmap

Language：Japanese  

本研究では,音声対話を通して自律型アンドロイドが与えられた社会的役割に適したキャラクタ表現方法を提案する.これまでに,キャラクタを構成する外向性,情緒不安定性,丁寧さという3要素と音声特徴量との関係性を調査した.印象評定の統計分析により,キャラクタ印象に影響する4つの音声特徴量が確認された.この結果を用いて,アンドロイドに与えられたキャラクタに合わせて発話量,相槌の頻度,フィラーの頻度,交替潜時の長さという4つの特徴量を制御するモデルを構築した.印象評価実験を行い,本モデルを用いることで,外向性と丁寧さの要素ついてはキャラクタ表現の可能性が確認された.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

人との自然な対話を指向したヒューマンロボットインタラクションにおいて,円滑なターンテイキングの実現は重要な課題である.円滑なターンテイキングを実現するためには,発話末における話者交替および継続の予測が必要である.また,これに関連するふるまいとして,相槌およびフィラーの生成が挙げられる.従来研究では,これらを予測する課題は独立に扱われてきた.本研究では,マルチタスク学習の枠組みにより,これらのふるまいの予測を同一のモデルで扱うことで,ターンテイキング予測の精度向上を目指す.自律型アンドロイドERICAを用いた対話コーパスでの実験により,提案手法の予測精度がシングルタスクの場合よりも向上することを示す.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

We address prediction of turn-taking considering related behaviors such as backchannels and fillers. Backchannels are used by the listeners to acknowledge that the current speaker can hold the turn. On the other hand, fillers are used by the prospective speakers to indicate a will to take a turn. We propose a turn-taking model based on multitask learning in conjunction with prediction of backchannels and fillers. The multitask learning of LSTM neural networks shared by these tasks allows for efficient and generalized learning, and thus improves prediction accuracy. Evaluations with two kinds of dialogue corpora of human-robot interaction demonstrate that the proposed multitask learning scheme outperforms the conventional single-task learning.

DOI： 10.21437/Interspeech.2018-1442

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/interspeech/interspeech2018.html#HaraITK18

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

本稿では,ユーザに共感し,ユーザの発話を促す傾聴対話システムについて述べる.近年,人間と雑談する対話システムの研究が盛んである.そのような対話システムが傾聴を行い交流支援を行うことで,ユーザの認知機能の維持に役立つと考えられる.本研究では,傾聴対話で必要な聞き手応答をアンドロイドERICAを用いた対話収録実験のコーパスから分析する.その結果,5種類の聞き手応答「掘り下げ質問」,「自分語り」,「評価応答」,「語彙的応答」が必要であることが判明した.それらを文脈に沿って生成し,また傾聴として適切な応答を1つ統計的に選択し出力する傾聴対話システムを提案する.その後,被験者実験による定性的評価を行う.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

音声対話システムとの対話をより自然で人間らしいものにするためには,人間が内面に持つ感情的な要素をシステムが獲得し,表現することが効果的であると考えられる.中でも「好感」は,対話への積極性や態度に大きく影響する重要な要素である.本研究では,自律型アンドロイドERICAを用いた初対面対話練習システムを構築するため,「発話量」「韻律」「視線」「笑い」「相槌」「頷き」などの特徴量から推定される「話題への興味」や「対話エンゲージメント」などを元に,アンドロイドが対話相手に抱く好感を生成し,発話に含む要素を選択することによって好感を表現することを試みる.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English  

CiNii Books

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)   Participation form：Joint(The vice charge)  

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computational Linguistics  

Attentive listening systems are designed to let people, especially senior people, keep talking to maintain communication ability and mental health. This paper addresses key components of an attentive listening system which encourages users to talk smoothly. First, we introduce continuous prediction of end-of-utterances and generation of backchannels, rather than generating backchannels after end-point detection of utterances. This improves subjective evaluations of backchannels. Second, we propose an effective statement response mechanism which detects focus words and responds in the form of a question or partial repeat. This can be applied to any statement. Moreover, a flexible turn-taking mechanism is designed which uses backchannels or fillers when the turn-switch is ambiguous. These techniques are integrated into a humanoid robot to conduct attentive listening. We test the feasibility of the system in a pilot experiment and show that it can produce coherent dialogues during conversation.

DOI： 10.18653/v1/w17-5516

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/sigdial/2017

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

Fillers are involved in the ease of understanding by listeners and turn-taking. However, the knowledge about its prosodic features is insufficient, and its modeling has not been done either. For these reasons, there is insufficient knowledge to generate natural and appropriate fillers in a dialog system at present. Therefore, for the purpose of clarifying the prosodic features of fillers, its relationship with occurrence positions or forms were analyzed in this research. 'Ano' and 'Eto' were used as forms, non-/boundary of Dialog Act and non-/turntaking for occurrence positions. Duration, F0, and intensity were utilized as prosodic features. As a result, the followings were found out: the prosodic features are different depending on the difference of the occurrence positions even for fillers of the same form, and similar prosodic features are found between the same occurrence positions even in different forms.

DOI： 10.21437/Interspeech.2017-1006

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/interspeech/2017

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

人との自然なインタラクションの実現を指向する自律型アンドロイドにおいて，人間どうしの対話でみられる自然なタイミングでの笑いの実現は重要な要素である．また，笑いの中でも，一方が笑うともう一方がつられて笑うshared laughterは対話の場を和ませたり，盛り上げたりするなどの効果があると考えらえる．本稿では，人間とアンドロイドERICAとの1対1対話において，アンドロイドはいかなる場合に人間につられて笑うべきかという観点で，言語および韻律特徴量を分析する．また分析結果に基づき人間が笑った場合にアンドロイドがつられて笑うべきか否かの予測を試みる.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

本稿では，傾聴対話システムにおける多様な応答の種類の選択や内容について述べる．近年，人間がロボットやエージェントと対話する機会が増えており，その中でも人間と自然な雑談を行えるシステムの研究が盛んである．本研究では，より自然な傾聴対話システムを構築するため，音声状態のユーザの発話を入力として，それに対する「掘り下げ質問」，「繰り返し応答」，「語彙的応答」，「自分語り」，「評価応答」の全ての応答を生成し，そのうちから文脈や先行発話の特徴を基に，統計的に適切な応答を1つ選択し，出力するシステムを提案する．

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computing Machinery, Inc  

We address the annotation of engagement in the context of human-machine interaction. Engagement represents the level of how much a user is being interested in and willing to continue the current interaction. The conversational data used in the annotation work is a human-robot interaction corpus where a human subject talks with the android ERICA, which is remotely operated by another human subject. The annotation work was done by multiple third-party annotators, and the task was to detect the time point when the level of engagement becomes high. The annotation results indicate that there are agreements among the annotators although the numbers of annotated points are different among them. It is also found that the level of engagement is related to turn-taking behaviors. Furthermore, we conducted interviews with the annotators to reveal behaviors used to show a high level of engagement. The results suggest that laughing, backchannels and nodding are related to the level of engagement.

DOI： 10.1145/3011263.3011271

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/icmi/2016ma3hmi

Language：Japanese   Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computing Machinery, Inc  

In the human-human first meeting dialogue, people tend to have a chat before their main topics to break tension or the "ice." This phenomenon is called "ice-breaking." For realizing this kind of natural conversations in dialogue systems, we address prediction of icebreaking using prosodic features in dialogue. This will allow for the systems to change conversation topics smoothly. At first, we statistically analyze relationships between prosodic features and icebreaking events, to select the useful feature sets showing significant effects. Then, prediction of ice-breaking is conducted by a logistic regression model with these features, which shows a promising result.

DOI： 10.1145/3005467.3005472

Scopus

CiNii Books

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icmi/assp4mi2016.html#InagumaINTK16

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：English   Publishing type：Research paper (scientific journal)   Publisher：CAMBRIDGE UNIV PRESS  

Conversations in poster sessions in academic events, referred to as poster conversations, pose interesting, and challenging topics on multi-modal signal and information processing. We have developed a smart posterboard for multi-modal recording and analysis of poster conversations. The smart posterboard has multiple sensing devices to record poster conversations, so we can review who came to the poster and what kind of questions or comments he/she made. The conversation analysis incorporates face and eye-gaze tracking for effective speaker diarization. It is demonstrated that eye-gaze information is useful for predicting turn-taking and also improving speaker diarization. Moreover, high-level indexing of interest and comprehension level of the audience is explored based on the multi-modal behaviors during the conversation. This is realized by predicting the audience's speech acts such as questions and reactive tokens.

DOI： 10.1017/ATSIP.2016.2

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

In human-human dialogue, especially in attentive listening such as counseling, backchannels play an important role. Appropriately coordinated backchannels will not only make smooth communication but also help establish rapport. By collecting counseling dialogue, we investigate whether and how synchrony is expressed by prosodic and linguistic features of backchannels with respect to the preceding speaker's utterances. First, we find out correlation patterns according to the type of backchannels and prosodic features; a larger correlation is observed for reactive tokens than acknowledging tokens and for the power features than the pitch features. Next, we investigate the relationship between the morphological complexity of backchannels and the syntactic complexity of the preceding clause/sentence unit. The result can be useful for generating a variety of backchannels adaptive to the speaker's utterances.

DOI： 10.1109/APSIPA.2015.7415301

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/apsipa/apsipa2015.html#KawaharaYUYT15

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

In human-human dialogue, especially in attentive listening such as counseling, backchannels are important not only for smooth communication but also for establishing rapport. Despite several studies on when to backchannel, most of the current spoken dialogue systems generate the same pattern of backchannels, giving monotonous impressions to users. In this work, we investigate generation of a variety of backchannel forms according to the dialogue context. We first show the feasibility of choosing appropriate backchannel forms based on machine learning, and the synergy of using linguistic and prosodic features. For generation of backchannels, a framework based on a set of binary classifiers is adopted to effectively make a "not-to-generate" decision. The proposed model achieved better prediction accuracy than a baseline which always outputs the same backchannel form and another baseline which randomly generates backchannels. Finally, evaluations by human subjects demonstrate that the proposed method generates backchannels as naturally as human choices, giving impressions of understanding and empathy.

DOI： 10.21437/Interspeech.2016-118

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/interspeech/interspeech2016.html#KawaharaYITW16

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

本稿では, 傾聴対話システムにおける多様な応答の種類の選択や内容について述べる. 近年, 人間がロボットやエージェントと対話する機会が増えており, 人間と自然な雑談を行えるシステムの研究が盛んである. 本研究では, 傾聴に特化した対話システムを構築するため, ユーザの発話に対して「語彙的応答」, 「評価応答」, 「繰り返し応答」, 「掘り下げ質問」のうち適切な返答の種類を選択し, 先行発話の焦点をずらさずに応答するシステムを提案する.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

通常の初対面対話では，本題に入る前に簡単な雑談を挟むことで場を和ませるアイスブレーキングという現象がしばしば見受けられる．近年，人とアンドロイドとの対話に関する研究は数多くなされているが，対話の開始時に着目した研究はほとんどない．本研究では，初対面対話という状況を前提として，WOZ方式で収録した対話コーパスを用いて，音響や画像などから得られるマルチモーダルな特徴量から場の和みについて分析を行い，自動的に場の和みを検出して雑談から本題に話題を自然に切り替えるシステムの開発を目的とする．

CiNii Books

J-GLOBAL

researchmap

Publishing type：Research paper (scientific journal)   Publisher：Japanese Society for Artificial Intelligence  

DOI： 10.1527/tjsai.c-g31

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese  

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (bulletin of university, research institution)   Participation form：Joint(The vice charge)  

Language：Japanese  

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

We propose multi-modal speaker diarization using acoustic and eye-gaze information in poster conversations. Eye-gaze infor- mation plays an important role in turn-taking, thus it is useful for predicting speech activity. In this paper, a variety of eye- gaze features are elaborated and combined with the acoustic information by the multi-modal integration model. Moreover, we introduce another model to detect backchannels, which in- volve different eye-gaze behaviors. This enhances the diariza- tion result by filtering meaningful utterances such as questions and comments. Experimental evaluations in real poster sessions demonstrate that eye-gaze information contributes to improve- ment of diarization accuracy under noisy environments, and its weight is automatically determined according to the Signal-to- Noise Ratio (SNR).

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/interspeech/2015

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publisher：一般社団法人情報処理学会  

学会やオープンラボなどでなされるポスターセッションにおける会話を対象として,各会話参与者がいつ発話したかという情報 (話者区間) とそのうちの相槌を検出する手法を述べる.実際のポスター会話では,自然な話し言葉や周囲の騒音などにより検出精度が低下する.著者らは話者区間検出において,従来法で用いられてきた音響情報に対して,会話における発話権取得で重要な役割を担う視線情報を統合するマルチモーダルな手法を提案している.本稿では,視線特徴量と確率的統合モデルについて検討・改善を行った.また,検出した聴衆の発話区間が相槌であるかを,話者区間検出と同様のマルチモーダルな手法で判定し,相槌を発話区間から除去することで質問やコメントなどの発話のみを抽出する.実験結果から,音響情報と視線情報を統合することで雑音環境下での検出精度の向上が確認された.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

雑談型音声対話システムの一つにユーザの話を聞く傾聴システムがある。従来の傾聴システムは質問を行う、相槌を打つなどといった基本的な行為が実装されているが、人間の対話でみられるような相槌の形態や韻律を調整することは行われていない。一方、相槌生成のタイミング予測の研究はあるが多様な形態の相槌をうつような枠組みは検討されていない。そこで本研究では、傾聴対話システムの実現を目標として相槌の形態パターンと先行発話の構文的な区切り、節境界の強弱との関係の分析を行う。

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

researchmap

Language：Japanese   Publishing type：Doctoral thesis  

Language：English   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (other academic)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Publishing type：Research paper (international conference proceedings)   Publisher：CEUR-WS.org  

This paper proposes how to enhance sharing knowledge among science communicators (SCs) in a museum. Our approach is to use SCROLL system to share learning log, which is defined as a digital record of what learners have learned in the daily life using ubiquitous technologies. One of the important skills of SCs is to create mutual communication with visitors and motivate them to know more. However, they have to acquire this skill from the daily experiences because this skill is not shared among SCs. Also, this skill is not easily taken over because the employment term of SCs is 5 years. In this situation, SCs registers learning logs into SCROLL and their activities are also recorded into video. They have a reflection session using learning logs and video in a week. Both interaction analysis by the expert and data analysis using SCROLL are combined.

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/lak/2014w

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

In this paper, we try to contribute to the design of future technologies used in science museums where there is no explicit, pre-determined relationship regarding knowledge between Science Communicators (SCs) and visitors. We illustrate the practice of interaction between them, especially focusing on social encounter. Starting in October 2012, we conducted a field study at the National Museum of Emerging Science and Innovation (Miraikan) in Japan. Based on multimodal analysis, we examine various activities, focusing on how expert SCs communicate about science: how they begin interactions with visitors, how they maintain them, and how they conclude them. © 2014 Springer International Publishing.

DOI： 10.1007/978-3-319-07440-5_59

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2014-5.html#BonoOTJ14

Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Publishing type：Research paper (international conference proceedings)  

This paper describes how to use a ubiquitous learning log system called SCROLL (System for Capturing and Reusing Of Learning Log) in a museum, especially, in order to support science communicators (SC). Ubiquitous Learning Log (ULL) is defined as a digital record of what you have learned in the daily life using ubiquitous technologies. It allows you to log your learning experiences with photos, audios, videos, location, QR-code, RFID tag, and sensor data, and to share and to reuse ULL with others. Using SCROLL, you can receive personalized quizzes and answers for your questions. This paper describes how to support science communicators in a science museum by using SCROLL, and shows the role of ULL to integrate the quantitative and qualitative analysis.

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ASSOC COMPUTING MACHINERY  

Communicative hand gestures play important roles in face-to-face conversations. These gestures are arbitrarily used depending on an individual; even when two speakers narrate the same story, they do not always use the same hand gesture (movement, position, and motion trajectory) to describe the same scene. In this paper, we propose a framework for the classification of communicative gestures in small group interactions. We focus on how many times the hands are held in a gesture and how long a speaker continues a hand stroke, instead of observing hand positions and hand motion trajectories. In addition, to model communicative gesture patterns, we use nonverbal features of participants addressed from participant gestures. In this research, we extract features of gesture phases defined by Kendon (2004) and co-occurring nonverbal patterns with gestures, i.e., utterance, head gesture, and head direction of each participant, by using pattern recognition techniques. In the experiments, we collect eight group narrative interaction datasets to evaluate the classification performance. The experimental results show that gesture phase features and nonverbal features of other participants improves the performance to discriminate communicative gestures that are used in narrative speeches and other gestures from 4% to 16%. © 2013 ACM.

DOI： 10.1145/2522848.2522898

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icmi/icmi2013.html#OkadaBTSN13

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ELSEVIER SCIENCE BV  

Conversational interactions contribute not only to the sharing of information and establishment of consensus but also to the formation and maintenance of mutual trust among conversational participants. We introduce a notion of concern alignment to describe the process of mutual trust formation in consensus-building dialogues. We propose a conceptual picture of dialogue processes taking place toward concern alignment and trust formation, and provide a qualitative analysis of health guidance dialogues between nurses and patients who were diagnosed as having metabolic syndrome to support the picture. We argue, based on our dialogue analysis result and other research in psychological and anthropological fields, that sharing of knowledge on values and preferences of others negotiated through concern alignment makes it possible for people to choose their actions based on trust on their partners. (C) 2013 The Authors. Published by Elsevier Ltd.

DOI： 10.1016/j.sbspro.2013.10.254

Web of Science

researchmap

Language：English   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

We address the estimation of the interest and comprehension level of an audience in poster sessions. Compared to lecture presentations, the audience's behaviors such as gazing and backchannels are more observable in poster presentations. These multi-modal behaviors are presumably related with their interest and comprehension level. We also assume that the interest and comprehension level can be judged by particular speech acts of the audience such as questions and reactive tokens. First, we make a preliminary analysis on their correlation. Next, we investigate the relationship between the audience's behaviors and the question type. Then, we conduct prediction of questions and their type based on the multi-modal behaviors during the relevant topic segment. Experimental results show that verbal backchannels and eye-gaze patterns are good predictors to this task, and also the combination of the multi-modal features is effective. Copyright © 2013 ISCA.

Web of Science

Scopus

CiNii Books

J-GLOBAL

researchmap

Other Link： https://dblp.uni-trier.de/conf/interspeech/2013

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA  

In this paper, we propose a new scheme for annotating response tokens (RTs) and their triggering expressions in Japanese multi-party conversations. In the proposed scheme, RTs are first identified and classified according to their forms, and then sub-classified according to their sequential positions in the discourse. To deeply study the contexts in which RTs are used, the scheme also provides procedures for annotating triggering expressions, which are considered to trigger the listener's production of RTs. RTs are classified according to whether or not there is a particular object or proposition in the speaker's turn for which the listener shows a positive or aligned stance. Triggering expressions are then identified in the speaker's turn; they include surprising facts and other newsworthy things, opinions and assessments, focus of a response to a question or repair initiation, keywords in narratives, and embedded propositions quoted from other's statement or thought, which are to be agreed upon, assessed, or noticed. As an illustrative application of our scheme, we present a preliminary analysis on the distribution of the latency of the listener's response to the triggering expression, showing how it differs according to RT's forms and positions.

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/lrec/2012

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

We investigate turn-taking behaviors in conversations in poster sessions. While the poster presenter holds most of the turns during sessions, the audience's utterances are more important and should not be missed. In this paper, therefore, prediction of turn-taking by the audience is addressed. It is classified into two sub-tasks: prediction of speaker change and prediction of the next speaker. We made analysis on eye-gaze information and its relationship with turn-taking, introducing joint eye-gaze events by the presenter and audience. We also parameterize backchannel patterns of the audience. As a result of machine learning with these features, it is found that combination of prosodic features of the presenter and the joint eye-gaze features is effective for predicting speaker change, while eyegaze duration and backchannels preceding the speaker change are useful for predicting the next speaker among the audience.

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/interspeech/2012

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：cognitivesciencesociety.org  

researchmap

Other Link： https://dblp.uni-trier.de/conf/cogsci/2012

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

researchmap

Publishing type：Research paper (international conference proceedings)  

In this paper, we propose a new annotation scheme for Japanese response tokens (RTs), which is based on strict and consistent procedures. Our scheme consists of two-stage annotation, in which RTs are first identified and classified according to their forms and then further sub-classified based on their sequential positions. Six forms are included in our class of RTs: i) responsive interjections, ii) expressive interjections, iii) lexical reactive expressions, iv) repetitions, v) completions, and vi) assessments. Some of them bear an additional tag according to their sequential position in the discourse: i) first pair parts, ii) second pair parts, iii) sequence-closing thirds, iv) other responding turns, and v) unclassifiable positions. We apply our scheme to annotate a Japanese three-party conversation corpus, and present the results of a preliminary analysis on the distribution of RTs in the corpus. © 2011 IEEE.

DOI： 10.1109/ICSDA.2011.6086001

Scopus

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INST SPEECH COMMUNICATION ASSOC  

We present a novel scheme for indexing "hot spots" in conversations, such as poster sessions, based on the reaction of the audience. Specifically, we focus on laughters and non-lexical reactive tokens, which are presumably related with funny spots and interesting spots, respectively. A robust detection method of these acoustic events is realized by combining BIC-based segmentation and GMM-based classification, with additional verifiers for reactive tokens. Subjective evaluations suggest that hot spots associated with reactive tokens are consistently useful while those with laughters are not so reliable. Furthermore, we investigate prosodic patterns of those reactive tokens which are closely related with the interest level. © 2010 ISCA.

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/interspeech/2010

Publishing type：Research paper (international conference proceedings)  

For effective indexing of presentation speech such as lectures and seminars, we explore a novel approach based on detection of the audience's interest level. In this work, we deal with poster presentations and focus on the backchannel responses or reactive tokens, which are frequently observed in poster conversations and presumably used for expressing the audience's interest level. First, we note that the most common reactive token “hai (yes)” is mainly used for acknowledging the speech segments, and that there are specific kinds of reactive tokens which can be used for expressing non-verbal information. Then, we made a prosodic analysis and identified effective combinations of the syllabic and prosodic patterns which express interest and surprise.

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：ISCA  

researchmap

Other Link： https://dblp.uni-trier.de/conf/diss/2010

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA  

In this paper, we propose a scheme for annotating utterance-level units in Japanese dialogs, which emerged from an analysis of the interrelationship among four schemes, i) inter-pausal units, ii) intonation units, iii) clause units, and iv) pragmatic units. The associations among the labels of these four units were illustrated by multiple correspondence analysis and hierarchical cluster analysis. Based on these results, we prescribe utterance-unit identification rules, which identify two sorts of utterance-units with different granularities: short and long utterance-units. Short utterance-units are identified by acoustic and prosodic disjuncture, and they are considered to constitute units of speaker's planning and hearer's understanding. Long utterance-units, on the other hand, are recognized by syntactic and pragmatic disjuncture, and they are regarded as units of interaction. We explore some characteristics of these utterance-units, focusing particularly on unit duration and syntactic property, other participants' responses, and mismatch between the two-levels. We also discuss how our two-level utterance-units are useful in analyzing cognitive and communicative aspects of spoken dialogs.

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/lrec/2010

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：English   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER-VERLAG BERLIN  

DOI： 10.1007/978-3-642-00609-8_23

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/jsai/jsai2008.html#IidaOT08

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.23.6_803

CiNii Books

researchmap

Other Link： http://id.nii.ac.jp/1004/00007093/

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.23.5_668

CiNii Books

researchmap

Other Link： http://id.nii.ac.jp/1004/00007059/

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The main charge)  

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

This article argues assessment sequence in ordinary conversation can be regarded as temporal expansion of social referencing. An ssessment sequence includes two different kinds of relationships at the same time. One is an agreement-demanding relation, which is a kind of prospective sequence like adjacency pair. The other is a commentary relation, which seems to be a retro-sequence. An assessment sequence also includes both a triadic relation similar to joint attention between two participants and the target of assessment and expressive dimension of communication.

CiNii Books

J-GLOBAL

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：The Association for Computer Linguistics  

An attempt was made to statistically estimate proposals which survived the discussion to be incorporated in the final agreement in an instance of a Japanese design conversation. Low level speech and vision features of hearer behaviors corresponding to aiduti, noddings and gaze were found to be a positive predictor of survival. The result suggests that non-linguistic hearer responses work as implicit proposal filters in consensus building, and could provide promising candidate features for the purpose of recognition and summarization of meeting events. © 2008 Association for Computational Linguistics.

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/sigdial/2008

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Ukrainian   Publishing type：Research paper (international conference proceedings)   Publisher：ISCA-INT SPEECH COMMUNICATION ASSOC  

A new project on multi-modal analysis of poster sessions is introduced. We have designed an environment dedicated to recording of poster conversations using multiple sensors, and collected a number of sessions, to which a variety of multi-modal information is annotated, including utterance units for individual speakers, backchannels, nodding, gazing, and pointing. Automatic speaker diarization, that is a combination of speech activity detection and speaker identification, is conducted using a set of distant microphones, and a reasonable performance is obtained. Then, we investigate automatic classification of conversation segments into two modes: presentation mode and question-answer mode. Preliminary experiments show that multi-modal features on non-verbal behaviors play a significant role in the indexing of this kind of conversations. Copyright © 2008 ISCA.

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/interspeech/2008

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

Detecting "Who spoke when?" from multi-party interactions such as meetings and poster presentations is valuable for adding metadata to the recordings or analyzing the discourse-structures of the multi-party interactions. To realize this function, we first detect speech periods from the observed signals (speech activity detection), and then classify the speech periods by its speakers (speaker diarization). In this paper, we adopt a speech activity detection method and a speaker diarization method to the recordings of poster sessions. The speech activity method performs based on the ratios of periodic and aperiodic components of observed signals. The speaker diarization method utilizes the direction of arrival estimation of the detected speech signals obtained from a microphone array. This paper reports preliminary results obtained from these methods.

CiNii Books

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)  

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.22.5_702

CiNii Books

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Publishing type：Research paper (scientific journal)  

This paper describes annotations for the Corpus of Spontaneous Japanese. The information we annotated to the corpus includes morphemes, clause units, dependency structures, summaries, and discourse structures. They are integrated in the form of XML. Morphological information was semi-automatically annotated to the transcribed text by reducing the human labor cost within the framework of morphological annotation that we proposed. Next, clause unites were detected based on the morphological information as basic units for our annotation. Then, dependency structures, summaries, discourse structures were annotated based on the clause units.

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

For robust detection of sentence and clause units in spontaneous speech such as lectures and meetings, we propose a novel cascaded chunking strategy which incorporates syntactic and semantic information. Application of general syntactic parsing is difficult for spontaneous speech having illformed sentences and disfluencies, especially for erroneous transcripts generated by ASR systems. Therefore, we focus on the local syntactic dependency of adjacent words and phrases, and train binary classifiers based on SVM (Support Vector Machines) for this purpose. An experimental evaluation using spontaneous talks of the CSJ (Corpus of Spontaneous Japanese) demonstrates that the proposed dependency analysis can be robustly performed and is effective for clause/sentence unit detection in ASR outputs. © 2007 IEEE.

DOI： 10.1109/ICASSP.2007.367179

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icassp/icassp2007.html#KawaharaST07

Publishing type：Research paper (international conference proceedings)  

We are constructing a research environment called the "IMADE room", which can capture a variety of multi-modal human interactions. With this setting, we have designed and conducted recordings of poster sessions made by one presenter and two audiences. In addition to speech data of individual participants, gazing, nodding, and pointing behaviors are recorded through multiple sensors. This article describes the specifications of the data collection and preliminary analyses of the relationship between verbal and non-verbal behaviors. Copyright 2007 ACM.

DOI： 10.1145/1330588.1330595

Scopus

researchmap

Language：Japanese   Publisher：Information Processing Society of Japan (IPSJ)  

Detecting "Who spoke when?" from multi-party interactions such as meetings and poster presentations is valuable for adding metadata to the recordings or analyzing the discourse-structures of the multi-party interactions. To realize this function, we first detect speech periods from the observed signals (speech activity detection), and then classify the speech periods by its speakers (speaker diarization). In this paper, we adopt a speech activity detection method and a speaker diarization method to the recordings of poster sessions. The speech activity method performs based on the ratios of periodic and aperiodic components of observed signals. The speaker diarization method utilizes the direction of arrival estimation of the detected speech signals obtained from a microphone array. This paper reports preliminary results obtained from these methods.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.22.6_838

CiNii Books

J-GLOBAL

researchmap

Other Link： http://id.nii.ac.jp/1004/00006861/

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.22.5_703

CiNii Books

J-GLOBAL

researchmap

Other Link： http://id.nii.ac.jp/1004/00006828/

Language：Japanese   Publisher：Information Processing Society of Japan (IPSJ)  

We are constructing a research environment called "IMADE room" which can capture a variety of multi-modal human interactions. With this setting, we have designed and conducted recordings of poster sessions by one presenter and two listeners. In addition to speech data of individual speakers, gazing, nodding and pointing events are recorded through multiple sensors. In this article, we describe the specification of the experiments and preliminary analysis on the relationship between verbal and non-verbal events.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

This paper describes the classification of attitudes in free conversations. In free conversations, sometimes a participant implicitly/explicitly represents his/her attitude just after a QA-pair. We pay attention to the questioner who shows an attitude, and we divide attitudes by the following parameters showing attitude with questioner's knowledge, forecasting the answer, object type and relation to other participant.

CiNii Books

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：European Language Resources Association (ELRA)  

In Japanese, syntactic structure of a sentence is generally represented by the relationship between phrasal units, or bunsetsus in Japanese, based on a dependency grammar. In the same way, the syntactic structure of a sentence in a large, spontaneous, Japanese-speech corpus, the Corpus of Spontaneous Japanese (CSJ), is represented by dependency relationships between bunsetsus. This paper describes the criteria and definitions of dependency relationships between bunsetsus in the CSJ. The dependency structure of the CSJ is investigated, and the difference in the dependency structures of written text and spontaneous speech is discussed in terms of the dependency accuracies obtained by using a corpus-based model. It is shown that the accuracy of automatic dependency-structure analysis can be improved if characteristic phenomena of spontaneous speech - such as self-corrections, basic utterance units in spontaneous speech, and bunsetsus that have no modifiee - are detected and used for dependency-structure analysis.

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/conf/lrec/2006

Language：Japanese   Publisher：Information Processing Society of Japan (IPSJ)  

The paper addresses chunking of spontaneous Japanese oriented for speech summarization and closed caption generation. Conventionally, inter-pause unit (IPU) has been a basic unit in annotation and processing of spontaneous speech, but pauses are not necessarily related with sentence or clause boundaries. On the other hand, automatic detection of sentence or clause boundaries based on machine learning techniques is not so reliable for speech recognition results. For robust chunking against ill-formedness in spontaneous speech and speech recognition errors, we focus on local bunsetsu dependencies. Combined with detection of predicates and classification of dependencies, we define a unit named "constituents", which apparently corresponds to subjects, predicates and case frames. With analysis and evaluation using the Corpus of Spontaneous Japanese (CSJ), it is shown that we can perform more reliable dependency structure analysis by focusing on adjacent bunsetsus and more robust detection of clause boundaries by extracting constituents.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publisher：Information Processing Society of Japan (IPSJ)  

This paper proposes a cascaded chunking method of spontaneous Japanese using bunsetsu dependency, pause and filler information. The proposed method generates two new units for spoken language processing : constituent and phrase. A constituent is generated by chunking bunsetsus using the adjacent bunsetsu dependency. A phrase is generated by chunking the constituents using pause and filler information. We made experiments using the Corpus of Spontaneous Japanese (CSJ). Most of constituents correspond to semantic chunks such as subjects, predicates and case frames, and 93% of clause boundaries is detected. More than half of phrases correspond to the clause boundaries, and cover 94% of sentence boundaries.

CiNii Books

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Publisher：電子情報通信学会  

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

This paper proposes a cascaded chunking method of spontaneous Japanese using bunsetsu dependency, pause and filler information. The proposed method generates two new units for spoken language processing : constituent and phrase. A constituent is generated by chunking bunsetsus using the adjacent bunsetsu dependency. A phrase is generated by chunking the constituents using pause and filler information. We made experiments using the Corpus of Spontaneous Japanese (CSJ). Most of constituents correspond to semantic chunks such as subjects, predicates and case frames, and 93% of clause boundaries is detected. More than half of phrases correspond to the clause boundaries, and cover 94% of sentence boundaries.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Society for Artificial Intelligence  

DOI： 10.11517/jjsai.20.4_370

CiNii Books

J-GLOBAL

researchmap

Other Link： http://id.nii.ac.jp/1004/00006281/

Language：Japanese   Publisher：人工知能学会  

CiNii Books

researchmap

Language：Japanese   Publisher：ことば工学事務局  

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

This paper describes the analysis of free conversation using multilevel description. Dialogue act tags are insufficient for description of non-task-oriented dialog. We suppose that dialog participants are going to find differences and common features of a sense of values in free conversation. We propose evaluation-level tags and the evaluation-level that expresses evaluation-side of utterance. Evaluation-level tags explain some features of sequence that cannot be explained by dialogue act tagging scheme.

CiNii Books

J-GLOBAL

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：ACM  

DOI： 10.1145/860575.860691

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/atal/aamas2003.html#NakanishiNITI03

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Participation form：Joint(The vice charge)  

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (bulletin of university, research institution)  

Language：Japanese  

CiNii Books

researchmap

Language：Japanese   Publishing type：Research paper (bulletin of university, research institution)  

Language：Japanese   Book type：Scholarly book

Responsible for pages：79-102   Language：Japanese   Book type：Scholarly book

Responsible for pages：115-125   Language：Japanese   Book type：Scholarly book

Responsible for pages：103-140   Language：Japanese   Book type：Scholarly book

Responsible for pages：167-186   Language：Japanese   Book type：Scholarly book

Total pages：182   Responsible for pages：47-60   Language：Japanese   Book type：Scholarly book

Total pages：182   Responsible for pages：99-100   Language：Japanese   Book type：Scholarly book

Responsible for pages：165-195   Language：Japanese   Book type：Scholarly book

Total pages：262p   Language：Japanese  

CiNii Books

researchmap

Language：Japanese   Book type：Scholarly book

Responsible for pages：191-217   Language：Japanese   Book type：Scholarly book

Total pages：xi, 259p   Language：Japanese  

CiNii Books

researchmap

Total pages：冊  

CiNii Books

researchmap

Total pages：xiii, 232p   Language：Japanese  

CiNii Books

researchmap

Language：Japanese   Book type：Scholarly book

Responsible for pages：278-296   Language：Japanese   Book type：Scholarly book

Language：Japanese   Book type：Scholarly book

Total pages：ix, 161p   Language：Japanese  

CiNii Books

researchmap

Responsible for pages：29-54   Language：Japanese   Book type：Scholarly book

Responsible for pages：55-75   Language：Japanese   Book type：Scholarly book

Responsible for pages：116-117   Language：Japanese   Book type：Scholarly book

Language：Japanese   Book type：Scholarly book

Total pages：viii, 445p   Language：Japanese  

CiNii Books

researchmap

Responsible for pages：22-34   Language：Japanese   Book type：Scholarly book

Responsible for pages：156-171   Language：Japanese   Book type：Scholarly book

Total pages：viii, 252p   Language：Japanese  

CiNii Books

researchmap

Responsible for pages：100-119   Language：Japanese   Book type：Scholarly book

Responsible for pages：67-102   Language：Japanese   Book type：Scholarly book

Responsible for pages：159-202   Language：Japanese   Book type：Scholarly book

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Association of Sociolinguistic Sciences  

DOI： 10.19024/jajls.21.2_92

CiNii Books

researchmap

Language：Japanese   Publisher：The Japanese Psychological Association  

DOI： 10.4992/pacjpa.83.0_SS-074

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Association of Sociolinguistic Sciences  

DOI： 10.19024/jajls.20.2_52

CiNii Books

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese  

我々は人と自然にインタラクションを行う自律型アンドロイドの研究開発を進めている。自律型アンドロイドは外見、振る舞い共に非常に人間に酷似させている。このため、ユーザはアンドロイドに対して何らかのキャラクタを感じとることが多い。そこで違和感のないキャラクタ表現を行うことで、より自然な対話を実現できると考えられる。本研究では、対話におけるアンドロイドの発話について、「発話量」「相槌」「フィラー」「交替潜時」といった振る舞いの特徴量を制御することによる「外向性」「情緒不安定性」「丁寧さ」といった特性への影響を実験により検証し、その結果に基づいたキャラクタ表現法を提案する。

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Association of Sociolinguistic Sciences  

DOI： 10.19024/jajls.19.1_1

CiNii Books

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japan Sociological Society  

This article aims to describe practices that help participants in talk-in-interactions share their images. In recent years, there has been growing interest in the collaborative nature of imagination. However, few attempts have been made at identifying the empirical practices that help participants share images. This article focuses on the practice of sharing images using both verbal and non-verbal behaviors, such as posture and gesture. Through conducting field work in a project team that is constructing a permanent science museum installation and detailed interaction analysis of the meeting interactions between members of the team, we find that reconciling a bodily orientation toward the depiction of images and a bodily orientation toward face-to-face interaction by verbal and non-verbal configurations of interaction enables participants to be both depictive in representing and explaining images and interactive in securing inter-subjectivity with recipients at any one time. We identified "head turning" behavior as a practice by which tellers can coordinate a bodily orientation toward description and face-to-face interaction. This practice enables participants in talk-in-interactions to describe members' verbal and non-verbal behavior as "sharing images" as a sociological description. Detailed examination of this practice shows how group members share their images in meeting interactions in order to work through organizational issues or problems.

DOI： 10.4057/jsr.66.39

CiNii Books

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：Japanese Cognitive Science Society  

This study examines the properties of gestures that serve to convey an image of the<br>speaker. The gesture observed can be perceived as a sub-type of gestures that linguistic<br>anthropologist Charles Goodwin termed "environmentally coupled gestures" in that the<br> meaning of the gesture is determined not by its semiotic property but by the mutual<br>elaboration of the gesture and other semiotic resources such as artifacts, utterance, and<br>gaze. A conversation analysis was conducted based on data collected during interaction<br>among members of a project team constructing an installation for a science museum.<br>The results show that team members recurrently re-do gestures that convey the image<br>of the speaker along with another gesture constructed to be coupled with the original<br>gesture.

DOI： 10.11225/jcss.22.557

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

In tele-auscultation, since a doctor cannot operate an auscultator by herself, she must refer a target point by a marker and have a helper in a distant place move an auscultator by proxy. This article analyzes a simulated tele-auscultation experiment and proposes an interactional pattern observed in the process of multimodal communication from pointing by marker by a doctor, operation of an auscultator by a helper to auscultation by the doctor. This pattern is then considered in terms of division of transmission between multi-channels of a system for tele-auscultation and a tele-conference system for conversation. At the end, problems on the system environment found in the experiment are addressed.

CiNii Books

J-GLOBAL

researchmap

Other Link： http://hdl.handle.net/2433/232631

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publisher：Japanese Cognitive Science Society  

This article reports field research on project meetings for developing an exhibition<br> at a science museum. These meetings were held in consecutive weeks over half a year. The<br> project team, consisting of members with diverse professions, collaborated on construct-<br> ing a novel exhibition. In the process of construction, problems discovered by a team<br> member often could not be solved individually. Therefore, members had to launch<br> collaborative problem-solving by relying upon interactional procedures, wherein one<br> member described a problem to the others. One of the procedures, which this article<br> examines, involves the usage of concern-introducing expressions, "kini-naru/suru-no-<br> wa/ga"(what the topic of concern is), appearing in approximately 30 meetings. These<br> expressions are examined in terms of their linguistic features. An analysis of sequential<br> relation of an utterance including this expression to the following utterances illustrates<br> that these expressions play a role for requesting other members to cope with solving the<br> problems presented. Finally, factors behind the use of concern-introducing expressions<br> are discussed in the light of members'professions and function of emotion.

DOI： 10.11225/jcss.22.84

CiNii Books

researchmap

J-GLOBAL

researchmap

Language：Japanese  

多人数会話のアーカイブを効率よく参照するためには，効果的なインデクス付与が要求される．本論文では，各々の会話参与者がいつ発話したかという情報（話者区間）とそのうちの相槌を同定する手法を述べる．多人数会話のマルチモーダルなセンシング環境として，我々はスマートポスターボードの構築を進めており，音響情報と視線情報を確率的に統合するマルチモーダルな手法を提案している．従来手法で用いられてきた音響情報だけでなく，視線情報も用いることで話者区間検出の音響的影響への頑健性，及び精度向上を確認した．また，相槌の同定についても視線情報の追加により精度が向上した．以上の処理結果を可視化し，多人数会話を効率的に視聴するためのブラウザを作成した．

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：Acoustical Society of Japan  

DOI： 10.20697/jasj.71.9_468

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：一般社団法人 人工知能学会  

<p>本研究は，日本科学未来館で日頃行われる対話を対象に，実世界における身体動作のセグメンテーション・コーディング手法を提案するものである．本手法は複数の研究領域で共有することを目指している．従来，身体動作のセグメンテーション・コーディングには，作業者間で動作の開始／終了時間，研究目的によるコーディングの一致が困難であるという問題が存在してきた．本発表では，これらの問題点を解決するための手法を議論する．</p>

DOI： 10.11517/pjsai.JSAI2015.0_3O12in

J-GLOBAL

researchmap

Language：Japanese  

DOI： 10.14943/66444

CiNii Books

researchmap

Other Link： http://hdl.handle.net/2115/56444

Language：Japanese   Publisher：茨城県立医療大学バランスオブライフ研究会  

CiNii Books

researchmap

Other Link： http://search.jamas.or.jp/link/ui/2014355847

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Psychological Association  

DOI： 10.4992/pacjpa.78.0_SS-041

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

For the purpose of proper understanding of multifaceted nature of hum an interaction, this article overvie ws recent trends in related domains and basic theories through comparison of spontaneous speech, dialogue corpus, multiparty conversation and real world interaction, and clarifies current issues for the domain.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

researchmap

researchmap

Language：Japanese  

DOI： 10.14943/61990

CiNii Books

researchmap

Other Link： http://hdl.handle.net/2115/52851

Language：Japanese   Publisher：明治書院  

CiNii Books

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：Human Interface Society  

<p>We assume that Science Communicators (SCs) have a social role; they interactively share their science knowledge with visitors. At National Museum of Emerging Science and Innovation (Miraikan), we investigate how visitors learn and experience practices of science with SCs. The purposes of this study are to build a theoretical framework to analyze interactions between SCs and visitors in a semi-opened space like museum, and to create a criterion to evaluate SCs' communication skills of sharing their scientific knowledge with visitors. In this paper, we introduce a research framework that we have used at Miraikan since October 2012 by showing procedures of analyzing one case that was conducted by a senior SC who has seven years and two months of experience. Additionally, we describe our preliminary analyses by using a methodology called, Conversation Analysis (CA). We choose nine interaction strategies in our analyses. These strategies are used to open up interactions with visitors. Then, we discuss whether or not nine other SCs use the same strategies. Since our results show the generality of the strategies, we are able to create a criterion of evaluating and developing SCs' skills. </p>

DOI： 10.11184/his.15.4_375

CiNii Books

researchmap

Language：Japanese   Publisher：ヒューマンインタフェース学会  

J-GLOBAL

researchmap

Language：Japanese   Publisher：一般社団法人情報処理学会  

Science Communicators (SC) have a social role which they convey their science knowledge to visitors in National Museum of Emerging Science and Innovation (Miraikan). How do visitors learn and experience science at the museum? The purpose of this study is building a framework to analyze their interactions which are organized by SC and visitors around exhibits and to evaluate their communication skills which SC convey their science knowledge to visitors. In this presentation, we introduce the research framework which we have started to apply at Miraikan from October 2012. And we discuss about...

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：人工知能学会  

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

The authors aim at clarifying how workshops are designed. For this purpose this presentation proposes a procedure for alignment between business workshop and consultation, both of which were held for the same sake of improvement of business plans. In detail, we extract the activities of the actual workshop, and annotate them with the segments of conversation of consultation. In the end, we build a corpus with that one can compare the activities of workshop with conversation segments of consultation.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

This study aims at developing a video viewing environment for immediate reflection on communication practices. It is desirable to provide participants with an opportunity for looking back (reflection) which facilitates the participants' awareness from the practice and improves their communication skills, as immediately as possible after a communication practice such as workshops, consultation and scientific communication. since the sooner after the practice, the better the participants remember behaviors or thoughts of themselves and of the other participants during the practice. This article proposes a method for constructing a video viewing environment which enables to browse video recordings effectively by using high-definition video and observation notes inputted by more than one observers with IRC chatting system. This article also introduces several cases in which we apply the method to immediate reflection on communication practices such as workshops and science cafes.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：Japan Society for Science Education  

DOI： 10.14935/jssep.37.0_110

CiNii Books

researchmap

Language：Japanese   Publisher：日本理科教育学会  

CiNii Books

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

This study focuses on the function of one body movement; head movement toward recipients during the course of performing others' future action. The result of the analysis shows that 1) turn around behavior projects that the performance will reach a boundary where response of recipients is required 2) turn around behavior contributes to manage both body orientation toward a performance and body orientation toward a interaction in which collaborative imagining is organized 3) timing of arise of turn around behavior is important in terms of their function in talk-in-interaction.

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

Meetings are important for promoting longitudinal group activities such as projects. Though transcriptions of such meetings may be a sort of discourse, it is insufficient for understanding them properly to analyze them merely as "closed" texts. This article indicates a direction for analyzing the transcriptions of meetings not as linguistic products but as actions embedded within a successive group activity.

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：一般社団法人 人工知能学会  

<p>「自分の身体」を出発点とする従来の多くの身体論に対して，本発表では，社会的インタラクションにおける身体の重要性はそれが他者にとって「見える」ものであるという点にあることを，日常会話とサッカーの事例分析に基づいて論じる．さらに，本質的に他者とのインタラクションに依存せざるを得ない動物としての人間にとって，この「見えるもの」としての身体という観点がどのような重要性を持っているかを考察する． </p>

DOI： 10.11517/pjsai.JSAI2012.0_3E2OS161

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：The Japanese Association of Sociolinguistic Sciences  

This paper introduced examples of gesture analysis by using a time-series data mining system. A time-series data mining system is able to reduce time and effort to annotate a large amount of nonverbal behavior data. At first, we obtain the trajectory of hand motion as multi-dimensional time-series data by using motion capture system. Second, each frame samples in the time-series data is classified to no motion patterns (such case that hand is set on home position) and motion patterns including gestures automatically. Third, frequent gesture patterns are extracted in time series by using motif (pattern) discovery algorithm. We applied these data mining methods to analyze gestures which are used in an explanation task. In the task, two participants explain the contents of animation to one participant. We collected datasets of 8 groups which are composed of 24 participants. Results of data mining show that amount of gestures in the task and timing to use gestures are different among groups and we can classify the timing in 8 groups to 3 types. From results of extracting of frequent gesture patterns, we show that 3 kind of gestures are used commonly in some groups.

DOI： 10.19024/jajls.15.1_38

CiNii Books

researchmap

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Language：Japanese   Publisher：Japanese Cognitive Science Society  

DOI： 10.11225/jcss.19.244

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books

J-GLOBAL

researchmap

Language：Japanese  

CiNii Books