Abstract: Malware is one of the big root causes of security incidents on the Internet. To fight against sophisticated malware, we have been developing Network Incident analysis Center for Tactical Emergency Response (NICTER). NICTER observes large-scale darknet (i.e., a set of unused IP addresses) and captures quite a few malware samples for grasping nationwide trends of malicious activities. Here we are facing technical challenges to deal with Cybersecurity big data in real-time manner. In this talk, we present an overview of NICTER and its spin-off technologies including a darknet monitoring-based alert system "DAEDALUS" and an anti-APT platform “NIRVANA-KAI". In addition, we explain our big data challenges behind the systems.

Abstract: The availability of large and comprehensive datasets plays a very important role in system security. For instance, large amount of data are often required to train accurate models, to observe global phenomena, to detect rare events, or to study attack patterns and campaigns. The same concept applies to many security areas ranging from spam to botnet detection, from malware analysis to network intrusion detection systems.