DanMcInerney/xsscrapy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
147 Commits 1 Branch 0 Tags
2d142844f4091a513fd6b382c96c9361c4f5bb40
Commit Graph

147 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dan McInerney
2d142844f4 Merge pull request #58 from shelld3v/patch-1 
Bug fix
 2022-01-14 10:31:31 -07:00
Dan McInerney
e1d8ff73fa Merge pull request #61 from fardeen-ahmed/master 
Updated a Critical Part in README file
 2022-01-14 10:30:58 -07:00
Fardeen Ahmed
3dbadfc78a Update README.md 2021-02-05 10:14:29 +00:00
shelld3v
b8def30f55 Bug fix 2020-12-27 10:41:50 +07:00
Dan McInerney
f6e65c0305 Merge pull request #33 from ddworken/master 
Added parameter to configure the bloom filter size
 2019-12-18 07:42:41 -07:00
Dan McInerney
06ad0aab01 fixed error "line 276, in getHostByName timeoutDelay = sum(timeout)" 2018-03-03 20:39:48 -07:00
David Dworken
a1617d088c Removed duplicate file 2016-11-11 18:15:13 -05:00
David Dworken
ff3e0f7a2a Merge branch 'bloomfilterSizes' 2016-11-11 18:13:10 -05:00
Dan McInerney
f859faa4e6 Merge pull request #32 from ddworken/master 
Switched to scrapy 1.1.0rc3 to fix SSL bug (Updated #31)
 2016-11-11 16:31:53 -05:00
David Dworken
b33060615e Updated logging levels to work with scrapy 1.1.0rc3 2016-11-10 22:39:24 -05:00
David Dworken
98fea8f4b9 Finished migration to 1.1.0rc3 
1. As of scrapy 1.1.0:
    > RedirectMiddleware now skips the status codes from handle_httpstatus_list on spider attribute or in Request‘s meta key (issue 1334, issue 1364, issue 1447).
    In order to allow xsscrapy to follow redirects, switched the range to not include 3xx redirect status codes
2. Explicitly include redirection information in the meta tags
3. Fixed a bug so that it always returns a valid object even when there is an error (so that it does not crash).
 2016-11-10 22:24:38 -05:00
David Dworken
0bd52ecd5e Added a bloomfilterSize parameter in settings.py to make it easier to adjust the size of the bloomfilters 
When scanning larger sites, xsscrapy can max out the size of the bloomfilter leading to the scanner stopping. Added a variable in xsscrapy/settings.py to make it easier to adjust the magnitude of the bloomfilter size.
 2016-10-24 21:32:05 -04:00
David Dworken
b199bcf981 Switched to scrapy 1.1.0rc3 to fix SSL bug 
Scrapy prior to 1.1.0rc3 had an issue where it would fail to connect to certain servers with an error stating ```<twisted.python.failure.Failure OpenSSL.SSL.Error: ('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert handshake failure'), ('SSL routines', 'SSL3_WRITE_BYTES', 'ssl handshake failure')>```. See https://github.com/scrapy/scrapy/issues/1764 for more info on this issue.

In that github issue, there were two potential solutions. Either add additional code forcing it to the correct code (see https://github.com/scrapy/scrapy/issues/1764#issuecomment-181950638). The other potential solution was upgrading to the latest release of scrapy.

There were pretty minimal changes needed to add support for 1.1.0rc3 (just changing import locations) so I went for that.
 2016-10-24 21:20:33 -04:00
Dan McInerney
763e4452bf Merge pull request #28 from ddworken/master 
Automatically detect script tags pointing to unregistered domains
 2016-07-29 17:39:13 -04:00
David Dworken
c616f66e61 Automatically detect script tags pointing to unregistered domains 2016-07-29 16:50:13 -04:00
Dan McInerney
e4e360d2c4 Merge pull request #27 from binarycanary/cookie_support 
Added cookie support
 2016-04-25 14:37:17 -06:00
Charlie
4bb032d53e Added cookie support 2016-04-17 02:52:12 -05:00
Dan McInerney
fb628efbf0 Merge pull request #24 from hslatman/clean-filename-solution 
Cleaner version for determining filename to output to
 2015-08-13 17:44:00 -06:00
Herman Slatman
cfa283ceaa Cleaner version for determining filename to output to 2015-08-14 01:27:29 +02:00
Dan McInerney
0b6c9994a6 Merge pull request #23 from tomberek/master 
More detailed requirements.txt and dependency documentation.
 2015-08-03 08:32:27 -06:00
Thomas Bereknyei
783f430da3 More detailed requirements.txt and dependency documentation. 2015-08-03 01:49:19 -04:00
Dan McInerney
4ec3e7d873 Merge pull request #21 from ddworken/master 
Need to import sys to use sys.exit()
 2015-02-12 20:11:36 -07:00
David Dworken
12dd23ab98 Need to import sys to use sys.exit() 2015-02-12 21:14:28 -05:00
Dan McInerney
a66cdd4596 removed debug line 2015-01-05 12:40:03 -07:00
Dan McInerney
22817decb4 minor 2014-12-19 08:42:22 -07:00
Dan McInerney
45758a9453 removed debug rule and set to handle all http codes not just 404 2014-12-18 15:45:58 -07:00
Dan McInerney
511a6a54c1 fixed 404 handling 2014-12-17 12:25:14 -07:00
Dan McInerney
318527271b Merge branch 'master' of https://github.com/DanMcInerney/xsscrapy 2014-12-17 12:16:43 -07:00
Dan McInerney
468bc082a9 fixed 404 crawling 2014-12-17 12:15:31 -07:00
Dan McInerney
a22f2e4132 Update xss_spider.py 2014-12-17 11:47:32 -07:00
Dan McInerney
9fe47ec97f Update README.md 2014-12-16 11:29:16 -07:00
Dan McInerney
67af0c6421 fixed reflected payload to accurately find semi colon 2014-12-15 00:05:52 -07:00
Dan McInerney
07c4b51d85 fixed unfiltered characters 2014-12-14 20:01:15 -07:00
Dan McInerney
80ee5c68d7 fixed delim length 2014-12-13 18:59:58 -07:00
Dan McInerney
ea5950bdeb added url path xss detection, refactored url param functions 2014-12-13 18:40:23 -07:00
Dan McInerney
60542c3429 removed JS escaped chars too early, also allowed empty params to be payloaded 2014-12-03 23:40:34 -07:00
Dan McInerney
f4061a6aa9 Merge pull request #19 from masterkorp/enhancement/typo-fix 
Fix typo
 2014-12-03 09:08:38 -07:00
Alfredo Palhares
4bad271e54 Fix typo 2014-12-03 16:03:11 +00:00
Dan McInerney
0070a91759 Merge branch 'master' of https://github.com/DanMcInerney/xsscrapy 2014-11-13 08:30:23 -07:00
Dan McInerney
2842ea1324 fixed a possible payload 2014-11-13 08:30:01 -07:00
Dan McInerney
8b42c37c7d Merge pull request #17 from ddworken/master 
Added rate limiting documentation.
 2014-11-10 09:10:48 -05:00
David Dworken
35c29e1e4e Added rate limiting documentation. 2014-11-10 07:55:11 -05:00
Dan McInerney
ce6587fe83 fixed rate limiting 2014-11-09 20:45:11 -05:00
Dan McInerney
c559d483e9 readded rate limiting 2014-11-09 20:28:37 -05:00
Dan McInerney
c07b0df0e1 attempts to fix broken form action URLs 2014-11-09 14:21:04 -05:00
Dan McInerney
94cf440bc1 shortened examples 2014-11-08 06:40:02 -05:00
Dan McInerney
62c813ac68 added -c option for max simultaneous connections 2014-11-08 06:35:52 -05:00
Dan McInerney
646ecf64b6 removal of comment 2014-11-08 05:11:19 -05:00
Dan McInerney
2f80a9f147 added / to fuzzer chars to prevent false+ within JS when trying to close the script tags with </script> 2014-11-07 15:38:58 -05:00
Dan McInerney
a2b9fdbd85 reduce false+ associated with gathering characters between two unrelated delimiter strings 2014-11-07 11:50:15 -05:00