Prabhjyot's Status Report #10 of 17

[Prabhjyot Singh Sodhi <prabhjyotsingh95 () gmail com>]

Hey list,

This week I finally started working on the multi-stage random forest
classifier.
Here it goes,

Accomplishments:
- Use Precision and Recall in addition to accuracy. I managed to calculate
the precision and recall for our test cases, but the results didn't seem to
help with the comparison of models. For the uninitiated, Precision and
recall in classifications context can be used to determine what kinds of
prints are being predicted correctly. (For example we might want popular
query prints say linux 4.0 to be predicted correctly compared to say a HP
printer. (Accuracy considers them alike.)
Having said that, the problem in our case is the low number of test prints.
Most of my test cases deal with 20/40 prints (total: 301). This makes it
difficult to compare precisions b/w groups since the number of test cases
per group are even lower.
- Started working on training the basic forests for the multi-stage
classifier. This includes adding a new target field which identifies how
are the prints classified in the first level. As of now we have started
with 5 classes of prints for the first stage: Linux, Windows, Macintosh,
BSD, Others
- Apart from this I wrote some more documentation trying to explain the
models. I will put these docs up soon for feedback.

Priorities:
- Get Prediction results from the multi-stage classifier
- tune the multi-stage classifier
- create a doc about the multi-stage classifier
- Read about converting the Makefile generated by cmake to be used by
autotools since opencv uses cmake while we'll need to integrate it with
autotools in nmap

Please pardon the delay in this week's report and feel free to write back
with any suggestions/ questions

Cheers,
Prabhjyot

_______________________________________________
Sent through the dev mailing list
https://nmap.org/mailman/listinfo/dev
Archived at http://seclists.org/nmap-dev/