Using OSINT sources for penetration testing

In early stages of a penetration test is a best practice the gathering of the most detailed information about the target, also using public data and search engines.

Below there is my short list of tools focused on information gathering


theHarvester

Developed by Christian Martorella, this tool gather emails, subdomains, hosts, employee names, open ports and banners from different public sources like search engines, PGP key servers and SHODAN computer database:

Passive discovery:

  • google: google search engine — www.google.com
  • google-profiles: google search engine, specific search for Google profiles
  • bing: microsoft search engine — www.bing.com
  • bingapi: microsoft search engine, through the API (you need to add your Key in the discovery/bingsearch.py file)
  • pgp: pgp key server — pgp.rediris.es
  • linkedin: google search engine, specific search for Linkedin users
  • shodan: Shodan Computer search engine, will search for ports and banner of the discovered hosts (http://www.shodanhq.com/)
  • vhost: Bing virtual hosts search

Active discovery:

  • DNS brute force: this plugin will run a dictionary brute force enumeration
  • DNS reverse lookup: reverse lookup of ip´s discovered in order to find hostnames
  • DNS TDL expansion: TLD dictionary brute force enumeration

[embed]https://github.com/laramies/theHarvester[/embed]


Snitch

This tool automate information gathering process for specified domain.

Using build-in dork categories, this tool helps gather specified informations domain which can be found using web search engines.

devil@hell:~/snitch$ python snitch.py
                       _ __       __  
           _________  (_) /______/ /_ 
          / ___/ __ / / __/ ___/ __  
         (__  ) / / / / /_/ /__/ / / /
        /____/_/ /_/_/__/___/_/ /_/ ~0.3   

Usage: snitch.py [options]

Options:
  -h, --help            show this help message and exit
  -U [url], --url=[url]
                        domain(s) or domain extension(s) separated by comma*
  -D [type], --dork=[type]
                        dork type(s) separated by comma*
  -C [dork], --custom=[dork]
                        custom dork*
  -O [file], --output=[file]
                        output file
  -S [ip:port], --socks=[ip:port]
                        socks5 proxy
  -I [seconds], --interval=[seconds]
                        interval between requests, 2s by default
  -P [pages], --pages=[pages]
                        pages to retrieve, 10 by default
  -v                    turn on verbosity

 Dork types:
  info   Information leak & Potential web bugs
  ext    Sensitive extensions
  docs   Documents & Messages
  files  Files & Directories
  soft   Web software
  all    All

[embed]https://github.com/Smaash/snitch[/embed]


Dmitry

DMitry has the ability to gather possible subdomains, email addresses, uptime information, tcp port scan, whois lookups (and more) about a host.

The information are gathered with following methods:

  • Perform an Internet Number whois lookup.
  • Retrieve possible uptime data, system and server data.
  • Perform a SubDomain search on a target host.
  • Perform an E-Mail address search on a target host.
  • Perform a TCP Portscan on the host target.
  • A Modular program allowing user specified modules

Example

The following command:

$ dmitry -iwns -o example.out google.com

creates a report named example.out, that looks like this:

HostIP:209.85.227.99
HostName:google.com
Gathered Inet-whois information for 209.85.227.99
---------------------------------
OrgName:    Google Inc. 
OrgID:      GOGL
Address:    1600 Amphitheatre Parkway
City:       Mountain View
StateProv:  CA
PostalCode: 94043
Country:    US
NetRange:   209.85.128.0 - 209.85.255.255 
CIDR:       209.85.128.0/17 
NetName:    GOOGLE
NetHandle:  NET-209-85-128-0-1
Parent:     NET-209-0-0-0-0
NetType:    Direct Allocation
NameServer: NS1.GOOGLE.COM
NameServer: NS2.GOOGLE.COM
NameServer: NS3.GOOGLE.COM
NameServer: NS4.GOOGLE.COM
Comment:    
RegDate:    2006-01-13
Updated:    2006-06-01
OrgTechHandle: ZG39-ARIN
OrgTechName:   Google Inc. 
OrgTechPhone:  +1-650-318-0200
OrgTechEmail:  arin-contact@google.com
# ARIN WHOIS database, last updated 2010-02-06 20:00
# Enter ? for additional hints on searching ARIN's WHOIS database.
#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at https://www.arin.net/whois_tou.html
Gathered Inic-whois information for google.com
---------------------------------
   Domain Name: GOOGLE.COM
   Registrar: MARKMONITOR INC.
   Whois Server: whois.markmonitor.com
   Referral URL: http://www.markmonitor.com
   Name Server: NS1.GOOGLE.COM
   Name Server: NS2.GOOGLE.COM
   Name Server: NS3.GOOGLE.COM
   Name Server: NS4.GOOGLE.COM
   Status: clientDeleteProhibited
   Status: clientTransferProhibited
   Status: clientUpdateProhibited
   Status: serverDeleteProhibited
   Status: serverTransferProhibited
   Status: serverUpdateProhibited
   Updated Date: 18-nov-2008
   Creation Date: 15-sep-1997
   Expiration Date: 14-sep-2011
>>> Last update of whois database: Sun, 07 Feb 2010 08:06:53 UTC <<<
NOTICE: The expiration date displayed in this record is the date the 
registrar's sponsorship of the domain name registration in the registry is 
currently set to expire. This date does not necessarily reflect the expiration 
date of the domain name registrant's agreement with the sponsoring 
registrar.  Users may consult the sponsoring registrar's Whois database to 
view the registrar's reported date of expiration for this registration.
TERMS OF USE: You are not authorized to access or query our Whois 
database through the use of electronic processes that are high-volume and 
automated except as reasonably necessary to register domain names or 
modify existing registrations; the Data in VeriSign Global Registry 
Services' ("VeriSign") Whois database is provided by VeriSign for 
information purposes only, and to assist persons in obtaining information 
about or related to a domain name registration record. VeriSign does not 
guarantee its accuracy. By submitting a Whois query, you agree to abide 
by the following terms of use: You agree that you may use this Data only 
for lawful purposes and that under no circumstances will you use this Data 
to: (1) allow, enable, or otherwise support the transmission of mass 
unsolicited, commercial advertising or solicitations via e-mail, telephone, 
or facsimile; or (2) enable high volume, automated, electronic processes 
that apply to VeriSign (or its computer systems). The compilation, 
repackaging, dissemination or other use of this Data is expressly 
prohibited without the prior written consent of VeriSign. You agree not to 
use electronic processes that are automated and high-volume to access or 
query the Whois database except as reasonably necessary to register 
domain names or modify existing registrations. VeriSign reserves the right 
to restrict your access to the Whois database in its sole discretion to ensure 
operational stability.  VeriSign may restrict or terminate your access to the 
Whois database for failure to abide by these terms of use. VeriSign 
reserves the right to modify these terms at any time.
The Registry database contains ONLY .COM, .NET, .EDU domains and
Gathered Netcraft information for google.com
---------------------------------
Retrieving Netcraft.com information for google.com
Netcraft.com Information gathered
Gathered Subdomain information for google.com
---------------------------------
Searching Google.com:80...
HostName:www.google.com
HostIP:209.85.227.99
Searching Altavista.com:80...
Found 1 possible subdomain(s) for host google.com, Searched 0 pages containing 0 results

[embed]https://www.aldeid.com/wiki/Dmitry[/embed]


wig — WebApp Information Gatherer

wig can identify numerous Content Management Systems and other administrative applications.

The application fingerprinting is based on checksums and string matching of known files for different versions of CMSes.

wig also tries to guess the operating system on the server based on the ‘server’ and ‘x-powered-by’ headers.

Example

$ python3 wig.py example.com
wig - WebApp Information Gatherer

Redirected to http://www.example.com
Continue? [Y|n]:
Scanning http://www.example.com...
_____________________________________________________ SITE INFO _____________________________________________________
IP                        Title                                                                                      
256.256.256.256           PAGE_TITLE                                 

______________________________________________________ VERSION ______________________________________________________
Name                      Versions                                               Type                                
Drupal                    7.38                                                   CMS                                 
nginx                                                                            Platform                            
amazons3                                                                         Platform                            
Varnish                                                                          Platform                            
IIS                       7.5                                                    Platform                            
ASP.NET                   4.0.30319                                              Platform                            
jQuery                    1.4.4                                                  JavaScript                          
Microsoft Windows Server  2008 R2                                                OS                                  

_____________________________________________________ SUBDOMAINS ____________________________________________________
Name                      Page Title                                             IP                                  
http://m.example.com:80   Mobile Page                                            256.256.256.257                     
https://m.example.com:443 Secure Mobil Page                                      256.256.256.258                     

____________________________________________________ INTERESTING ____________________________________________________
URL                       Note                                                   Type                                
/test/                    Test directory                                         Interesting                         
/login/                   Login Page                                             Interesting                         

_______________________________________________ PLATFORM OBSERVATIONS _______________________________________________
Platform                  URL                                                    Type                                
ASP.NET 2.0.50727         /old.aspx                                              Observation                         
ASP.NET 4.0.30319         /login/                                                Observation                         
IIS 6.0                   http://www.example.com/templates/file.css              Observation                         
IIS 7.0                   https://www.example.com/login/                         Observation                         
IIS 7.5                   http://www.example.com                                 Observation                         

_______________________________________________________ TOOLS _______________________________________________________
Name                      Link                                                   Software                            
droopescan                https://github.com/droope/droopescan                   Drupal                              
CMSmap                    https://github.com/Dionach/CMSmap                      Drupal                              

__________________________________________________ VULNERABILITIES __________________________________________________
Affected                  #Vulns                                                 Link                                
Drupal 7.38               5                                                      http://cvedetails.com/version/185744

_____________________________________________________________________________________________________________________
Time: 11.3 sec            Urls: 310                                              Fingerprints: 37580

[embed]https://github.com/jekyc/wig[/embed]


AngryFuzz3r

AngryFuzz3r is a collection of tools for pentesting to gather information and discover vulnerabilities of the targets based on Fuzzedb https://github.com/fuzzdb-project/fuzzdb project:

FuzzDB was created to increase the likelihood of causing and identifying conditions of security interest through dynamic application security testing. It’s the first and most comprehensive open dictionary of fault injection patterns, predictable resource locations, and regex for matching server responses.

The tool is developed by Iheb B.Salem.

Features

  • Fuzz url set from an input file
  • Concurrent relative path search
  • Configurable number of fuzzing workers
  • Fuzz CMS ==> Wordpress,Durpal,Joomla
  • Generate reports of the valid paths

Usage

$ python angryFuzzer.py -h
Usage: angryFuzzer.py [options]
Options:
  -h, --help            show this help message and exit
  -q, --quiet           Silent mode ,only reports
  -u URL, --url=URL      URL of the Target
  -c CMS, --cms=CMS     scan CMS ==> wp ,dp
  -w WORDLIST, --wordlist=WORDLIST
                        Custom wordlist

[embed]https://github.com/ihebski/angryFuzzer[/embed]