Building Intrusion Detection System but from where to begin

Question

I have searched a lot on Intrusion Detection system but now I am confused as now from where should I start. I dont know whether any open source reusable codes exists but I

Answer 1

Most intrusion detection systems which use Neural Networks make use of supervised training, ie. the system prompts you for an opinion when certain changes are requested to its host. I suggest that you start with finding out the methodology for hooking change requests. In windows that could involve using a system hook to filter certain actions that are requested by applications. This will allow your app the option of prompting you for a response, that response will overtime be fed into the neural net. This dataset then can be used to optimize the recognition of certain patterns and your responses to those patterns. There are obviously more things to consider when building a system such as this but you should be off to a good start based on what I said.

Answer 2

I study in the same subject. Intrusion detection and machine learning. It is rather broad subject. I will answer more about data pre-processing and feature construction point of view. Neural Network part is different story altogether.

First of all, this area is heavily commercialized therefore there is almost no open source code examples. A lot things are done commercially in a closed ecosystem.

From academic perspective: There is a big data set problem. DK99C (Darpa - KDD99 data set) exists but it is very old. KDD99 dataset is constructed from DARPA tcpdumps. They used bro IDS , tcpdump api to construct features. From my perspective it is a lot harder to create features from raw tcpdump than working with machine learning algorithms (Neural Network) on ready features.

Read this article to learn more about how it (KDD99) is constructed

Article (Lee2000framework) Lee, W. & Stolfo, S. J. 
A framework for constructing  features and models for intrusion detection systems 
ACM Trans. Inf. Syst. Secur., ACM, 2000, 3, 227-261

Read this article and its presentation to learn why this subject is a hard problem to study.

 Inproceedings (Sommer2010Outside) Sommer, R. & Paxson, V. 
 Outside the Closed World: On Using Machine Learning for Network Intrusion Detection
 Proceedings of the 2010 IEEE Symposium on Security and Privacy, IEEE Computer Society, 2010, 305-316

Read this article to see how most academics work in this subject. A bit disappointing really.

Article (Tavallaee2010Toward) Tavallaee, M.; Stakhanova, N. & Ghorbani, A. 
Toward Credible Evaluation of Anomaly-Based Intrusion-Detection Methods 
Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 2010, 40, 516 -524

Read this why DK99C is considered harmful. It is harmful but no other credible dataset exists.

Article (Brugger2007KDD) Brugger, S. 
KDD Cup’99 dataset (Network Intrusion) considered harmful 
KDnuggets newsletter, 2007, 7, 15

Read this about taxonomy of IDS data pre processing

Article (Davis2011Data) Davis, J. J. & Clark, A. J. 
Data preprocessing for anomaly based network intrusion detection: A review 
Computers & Security, 2011, 30, 353 - 375