Toward Imitating Visual Attention of Experts in Software Development Tasks

Expert programmers' eye-movements during source code reading are valuable sources that are considered to be associated with their domain expertise. We advocate a vision of new intelligent systems incorporating expertise of experts for software development tasks, such as issue localization, comm...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2019-03
Main Authors: Ikutani, Yoshiharu, Koganti, Nishanth, Hata, Hideaki, Kubo, Takatomi, Matsumoto, Kenichi
Format: Article
Language:English
Subjects:
Experts
Eye movements
Software
Software development
Source code
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Expert programmers' eye-movements during source code reading are valuable sources that are considered to be associated with their domain expertise. We advocate a vision of new intelligent systems incorporating expertise of experts for software development tasks, such as issue localization, comment generation, and code generation. We present a conceptual framework of neural autonomous agents based on imitation learning (IL), which enables agents to mimic the visual attention of an expert via his/her eye movement. In this framework, an autonomous agent is constructed as a context-based attention model that consists of encoder/decoder network and trained with state-action sequences generated by an experts' demonstration. Challenges to implement an IL-based autonomous agent specialized for software development task are discussed in this paper.
ISSN:2331-8422