PaulDotCom mailing list archives
Re: Python and asynchronous sockets question
From: Karl Schuttler <karl.schuttler () gmail com>
Date: Thu, 10 Jan 2013 12:45:31 -0500
On Thu, Jan 10, 2013 at 10:57 AM, allison nixon <elsakoo () gmail com> wrote:
Yeah that's why I said I accidentally wrote it in python. I just want to know if I can salvage this since I already wrote it.
I think you're misinterpreting David's comment - he is saying that threading is what you want. By launching each request in a new thread, you can spawn a whole bunch of requests off and not have to spend execution time waiting for the response to come back ("babysitting"). I wrote an HTTPS scanner to send HTTP POST to a list of IPs and try default usernames and passwords, using threads. Using threads I was able to do >500 requests in parallel, doing 11k hosts in about 5 minutes. Certainly try to find what is consuming your resources. Karl
I also just realized something that I might have been doing wrong that is hogging most of the resources... if I can get 200 requests per second with python i'll let you guys know... On Thu, Jan 10, 2013 at 10:37 AM, Martín <ole () makensi es> wrote:Hi! be also advised that because of the GlL [1] of the cpython interpreter, you may need to switch to an alternative interpreter implementation when performance of high concurrency applications is a must. In other words, Python may not be the best choice when trying to go to such performance limits. [1] http://en.wikipedia.org/wiki/Global_Interpreter_Lock On Thu, Jan 10, 2013 at 4:22 PM, Scott Kragen <skragen () gmail com> wrote:Allison, Have you looked into threading for python? I have used this library in several of projects because writing a thread pool from scratch started to give me a headache. http://code.activestate.com/recipes/577105-synchronization-decorator-for-class-methods/ The advantage of a thread pool is it can que based on the max amount of threads you define. Scott On Thu, Jan 10, 2013 at 10:06 AM, allison nixon <elsakoo () gmail com> wrote:Say I'm writing a broadscanner for a pet project and I accidentally wrote the entire thing in Python. Using asyncore and sockets, the best I can get is 3 http requests per second, maybe 10 per second at the very max. For a scanner this is of course, very lame. My goal is to of course make as many http request as a desktop computer will handle, so perhaps 200 per second with whatever number of sockets waiting in the background for a response(which I check periodically for a response and parse when I get it) Is there any way I can acheive 200 requests per second without learning another programming language? _______________________________________________ Pauldotcom mailing list Pauldotcom () mail pauldotcom com http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com-- ---------------------------------------------------------------------------------------------------------------------- There is only one metric in security that can be truly measured and that is failure! --- Jack Daniels ---------------------------------------------------------------------------------------------------------------------- _______________________________________________ Pauldotcom mailing list Pauldotcom () mail pauldotcom com http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com_______________________________________________ Pauldotcom mailing list Pauldotcom () mail pauldotcom com http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com-- _________________________________ Note to self: Pillage BEFORE burning. _______________________________________________ Pauldotcom mailing list Pauldotcom () mail pauldotcom com http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com
_______________________________________________ Pauldotcom mailing list Pauldotcom () mail pauldotcom com http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com
Current thread:
- Python and asynchronous sockets question allison nixon (Jan 10)
- Re: Python and asynchronous sockets question Scott Kragen (Jan 10)
- Re: Python and asynchronous sockets question allison nixon (Jan 10)
- Re: Python and asynchronous sockets question David Hoelzer (Jan 10)
- Re: Python and asynchronous sockets question Martín (Jan 10)
- Re: Python and asynchronous sockets question allison nixon (Jan 10)
- Re: Python and asynchronous sockets question Scott Kragen (Jan 10)
- Re: Python and asynchronous sockets question Karl Schuttler (Jan 10)
- Re: Python and asynchronous sockets question allison nixon (Jan 11)
- Re: Python and asynchronous sockets question allison nixon (Jan 10)
- Re: Python and asynchronous sockets question Scott Kragen (Jan 10)