Commit 6a111a15 authored by Lorenzo Faletra's avatar Lorenzo Faletra

Import Upstream version 3.1.1

parents

Too many changes to show.

To preserve performance only 388 of 388+ files are displayed.

server/www/script/* linguist-vendored
Please search the [Wiki](https://github.com/infobyte/faraday/wiki) for a solution before posting a ticket. Use the <strong>“New Support Request”</strong> button to the right of the screen to submit a ticket for technical support.
## Issue Type
- Bug Re port
- Feature Idea
- Documentation Report
## Faraday version
Paste the output of the *./faraday.py --version* command
## Component Name
If you know where the problem lays indicate it:
WebGui/GTKGui/Plugin/Console/Continuous Scanning/Etc.
## Steps to reproduce
Provide detailed steps on how the issue happened so we can try to reproduce it. If the issue is random, please provide as much information as possible.
## Expected results
What did you expect to happen when following the steps above?
### Debugging tracebacks (current results)
Try to reproduce the bug with the server and/or gtk client in debug mode and check the logs for the ERROR string.
Add here any errors you find while running in debug mode or, if possible, Faraday’s log files (located at *$HOME/.faraday/logs/*).
If you need help on how to execute in debug mode [click here for more information](https://github.com/infobyte/faraday/wiki/troubleshooting).
Please attach the result of:
pip freeze > requirements_freeze.txt
### Screenshots
If you don't find anything on the logs, please provide screenshots of the error.
## Environment information
### Configuration files
Mention any settings you have changed/added/removed.
### Reports/Extra data
If you are having issues with plugins, please attach relevant files if possible.
(strip your reports of all sensitive information beforehand).
### OS
Provide information on your operating system. Example:
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.10
DISTRIB_CODENAME=yakkety
DISTRIB_DESCRIPTION="Ubuntu 16.10"
*.py[cod]
# C extensions
*.so
# Packages
*.egg
*.egg-info
dist
build
eggs
parts
var
sdist
develop-eggs
.installed.cfg
lib
lib64
__pycache__
# Installer logs
pip-log.txt
# Unit test / coverage reports
.coverage
.tox
nosetests.xml
test_cases/.cache/
test_cases/htmlcov/
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
pushReports.py
*.swp
# Tests nodejs
node_modules
# Couch
.couchadmin
# Ignore faraday server pid
server/.faraday-server.pid
#Ignore images executive report
reports/executive/images/ease-of-resolution-piechart.png
reports/executive/images/impact-piechart.png
reports/executive/images/vulns-piechart.png
#Ignore visual studio code configs
typings/
jsconfig.json
.vscode/
# PyCharm project files
.idea
# vFeed DB
data/vfeed.db
# Images based on DB Schema
entity_dbschema.png
uml_schema.png
# Documentation builds
doc/_build/
The PRIMARY AUTHORS are:
* Daniel Foguelman
* Esteban Guillardoy
* Ezequiel Tavella
* Facundo de Guzmán
* Federico Kirschbaum
* Francisco Amato
* Franco Linares
* German Riera
* Joaquín López Pereyra
* Leonardo Lazzaro
* Martín Rocha
* Matias Ariel Ré Medina
* Matias Lang
* Micaela Ranea Sánchez
* Sebastian Kulesz
* Eric Horvat
Project contributors
* Alejandro Parodi
* Andrés López Luksenberg
* Andres Tarantini
* Brice Samulenok
* Buanzo
* csk
* dmknght
* Elian Gidoni
* Endrigo Antonini
* Federico Fernandez
* James Jara
* Javier aguinaga
* Jorge Luis Gonzalez Iznaga
* Juan Urbano
* Korantin Auguste
* Martin Tartarelli
* Mike Zhong (go bears)
* Necrose99
* Ronald Iraheta
* Roberto Focke
* Sliim
* Thierry Beauquier
* tsxltjecwb
* Ulisses Albuquerque
* xtr4nge
#!groovy
node (label: "master"){
def ENV_PATH = "$HOME/venv/faraday"
echo "${ENV_PATH}"
stage("Clean virtualenv") {
sh "rm -rf ${ENV_PATH}"
}
stage("Install Python Virtual Enviroment") {
sh "/usr/local/bin/virtualenv --no-site-packages ${ENV_PATH} --python=/usr/local/bin/python"
}
// Get the latest version of our application code.
stage ("Pull Code from SCM") {
checkout scm
}
stage ("Install Application Dependencies") {
sh """
source ${ENV_PATH}/bin/activate
pip install virtualenv responses
pip install -U -r $WORKSPACE/requirements.txt
pip install -U -r $WORKSPACE/requirements_server.txt
pip install -U -r $WORKSPACE/requirements_extras.txt
pip install -U -r $WORKSPACE/requirements_dev.txt
deactivate
"""
}
stage ("Check code style") {
sh """
sloccount --duplicates --wide --details $WORKSPACE | fgrep -v .git > $WORKSPACE/sloccount.sc || :
find $WORKSPACE -name \\"*.py\\" | egrep -v '^./tests/' | xargs pyflakes > $WORKSPACE/pyflakes.log || :
find $WORKSPACE -name \\"*.py\\" | egrep -v '^./tests/' | xargs pylint --output-format=parseable --reports=y > $WORKSPACE/pylint.log || :
eslint -c /home/faraday/.eslintrc.js -f checkstyle $WORKSPACE/server/www/scripts/**/* > eslint.xml || true
"""
warnings canComputeNew: false, canResolveRelativePaths: false, consoleParsers: [[parserName: 'PyFlakes']], defaultEncoding: '', excludePattern: '', healthy: '', includePattern: '', messagesPattern: '', parserConfigurations: [[parserName: 'AcuCobol Compiler', pattern: 'pyflakes.log']], unHealthy: ''
}
stage ("Run Unit/Integration Tests") {
def testsError = null
try {
sh """
source ${ENV_PATH}/bin/activate
cd $WORKSPACE && pytest -v --junitxml=$WORKSPACE/xunit.xml || :
deactivate
"""
step([$class: 'CoberturaPublisher', autoUpdateHealth: false, autoUpdateStability: false, coberturaReportFile: '**/coverage.xml', failNoReports: false, failUnhealthy: false, failUnstable: false, maxNumberOfBuilds: 0, onlyStable: false, sourceEncoding: 'ASCII', zoomCoverageChart: false])
}
catch(err) {
testsError = err
currentBuild.result = 'FAILURE'
}
finally {
junit "**/xunit.xml"
notifyBuild(currentBuild.result, "SQLite Build")
if (testsError) {
throw testsError
}
}
}
stage ("Run Unit/Integration Tests (with PostgreSQL)") {
def testsError = null
try {
withCredentials([string(credentialsId: 'postgresql_connection_string', variable: 'CONN_STRING')]) {
sh """
source ${ENV_PATH}/bin/activate
cd $WORKSPACE && pytest -v --junitxml=$WORKSPACE/xunit-postgres.xml --connection-string "$CONN_STRING" || :
deactivate
"""
step([$class: 'CoberturaPublisher', autoUpdateHealth: false, autoUpdateStability: false, coberturaReportFile: '**/coverage.xml', failNoReports: false, failUnhealthy: false, failUnstable: false, maxNumberOfBuilds: 0, onlyStable: false, sourceEncoding: 'ASCII', zoomCoverageChart: false])
}
}
catch(err) {
testsError = err
currentBuild.result = 'FAILURE'
}
finally {
junit "**/xunit-postgres.xml"
notifyBuild(currentBuild.result, "PostgreSQL Build")
if (testsError) {
throw testsError
}
}
}
stage ("Build docs") {
sh """
source ${ENV_PATH}/bin/activate
pip install sphinx
mkdir -p ~/docs
rm -rf ~/docs/jenkins_build
cd $WORKSPACE/doc && make html && cp -r _build/html ~/docs/jenkins_build
"""
}
stage ("Run Closure Compiler") {
try {
sh """
java -jar /home/faraday/closure-compiler-v20180610.jar $WORKSPACE/server/www/scripts
"""
}
catch (err) {
currentBuild.result = 'FAILURE'
}
finally {
notifyBuild(currentBuild.result, "Closure compiler")
}
}
}
def notifyBuild(String buildStatus = 'STARTED', String extraMessage = '') {
// build status of null means successful
buildStatus = buildStatus ?: 'SUCCESSFUL'
// Default values
def colorName = 'RED'
def colorCode = '#FF0000'
def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
def summary = "${subject} (${env.BUILD_URL}) " + extraMessage
// Override default values based on build status
if (buildStatus == 'STARTED') {
color = 'YELLOW'
colorCode = '#FFFF00'
} else if (buildStatus == 'SUCCESSFUL') {
color = 'GREEN'
colorCode = '#00FF00'
} else {
color = 'RED'
colorCode = '#FF0000'
summary = summary + ' @channel'
}
// Send notifications
slackSend (color: colorCode, message: summary)
}
## About
Faraday introduces a new concept - IPE (Integrated Penetration-Test Environment) a multiuser Penetration test IDE. Designed for distributing, indexing, and analyzing the data generated during a security audit.
> Made for true pentesters!
Faraday was made to let you take advantage of the available tools in the community in a truly multiuser way.
Designed for simplicity, users should notice no difference between their own terminal application and the one included in Faraday. Developed with a specialized set of functionalities, users improve their own work. Do you remember the last time you programmed without an IDE? What IDEs are to programming, Faraday is to pentesting.
![GUI - GTK](https://raw.github.com/wiki/infobyte/faraday/images/client/gtk_main_window.png)
Faraday crunches the data you load into different visualizations that are useful to managers and pentesters alike.
![GUI - Web](https://raw.github.com/wiki/infobyte/faraday/images/dashboard/dashboard.png)
To read about the latest features check out the [release notes](https://github.com/infobyte/faraday/blob/master/RELEASE.md)!
## Getting Started!
Check out our documentacion for datailed information on how to install Faraday in all of our supported platforms:
![Supported Os](https://raw.github.com/wiki/infobyte/faraday/images/platform/supported.png)
To begin the instalation process check our out [First Step](https://raw.github.com/wiki/infobyte/faraday/First-steps) Wiki.
## New Features!
All of Faraday's latest features and updates are always available on our [blog](http://blog.infobytesec.com/search/label/english).
There are new entries every few weeks, don't forget to check out our amaizing new improvements on it's last entry!
## Plugins list
You feed data to Faraday from your favorite tools through Plugins. Right now there are more than [60+ supported tools](https://github.com/infobyte/faraday/wiki/Plugin-List), among which you will find:
![](https://raw.github.com/wiki/infobyte/faraday/images/plugins/Plugins.png)
There are three Plugin types: **console** plugins which intercept and interpret the output of the tools you execute, **report** plugins which allows you to import previously generated XMLs, and **online** plugins which access Faraday's API or allow Faraday to connect to external APIs and databases.
[Read more about Plugins](http://github.com/infobyte/faraday/wiki/Plugin-List).
## Features
### Workspaces
Information is organized into various **Workspaces**. Each Workspace contains a pentest team's assignments and all the intel that is discovered.
### Conflicts
If two plugins produce clashing information for an individual element, a conflict that the user will have to resolve is generated. An example is if **user1** incorporates host *127.0.0.1 OS:Linux* and **user2** incorporates *127.0.0.1 OS: Linux Ubuntu 13.10*.
On our [GTK interface](https://github.com/infobyte/faraday/wiki/Usage#gtk-gui) there's a button on the bottom right corner of the main window displaying the number of conflicts in the current workspace. To resolve them, just click on the button and a window will open where you can edit the conflicting objects and select which one to keep.
### Faraday plugin
Using our plugin you can perform various actions using the command line, for example:
$ cd faraday-dev/bin/
$ ./fplugin create_host 192.154.33.222 Android
1a7b2981c7becbcb3d5318056eb29a58817f5e67
$ ./fplugin filter_services http ssh -p 21 -a
Filtering services for ports: 21, 22, 80, 443, 8080, 8443
192.168.20.1 ssh [22] tcp open None
192.168.20.1 http [443] tcp open None
192.168.20.7 ssh [22] tcp open Linux
192.168.20.7 http [443] tcp open Linux
192.168.20.11 ssh [22] tcp open Linux
Read more about the [Faraday Plugin](https://github.com/infobyte/faraday/wiki/faraday-plugin).
### Notifications
Updating objects on other Faraday instances result in notifications on your
Faraday GTK Client.
![](https://raw.github.com/wiki/infobyte/faraday/images/client/gtk_notifications_dialog.png)
### CSV Exporting
Faraday supports CSV Exporting from its WEB UI.
[More information](Exporting-the-information)
## Links
* Homepage: https://www.faradaysec.com
* User forum: https://forum.faradaysec.com
* User's manual: https://github.com/infobyte/faraday/wiki
* Download: [.tar.gz](https://github.com/infobyte/faraday/tarball/master)
* Commits RSS feed: https://github.com/infobyte/faraday/commits/master.atom
* Issue tracker: https://github.com/infobyte/faraday/issues
* Frequently Asked Questions (FAQ): https://github.com/infobyte/faraday/wiki/FAQ
* Mailing list subscription: https://groups.google.com/forum/#!forum/faradaysec
* Twitter: [@faradaysec](https://twitter.com/faradaysec)
* [Demos](https://github.com/infobyte/faraday/wiki/Demos)
* IRC: [ircs://irc.freenode.net/faraday-dev](ircs://irc.freenode.net/faraday-dev) [WebClient](https://webchat.freenode.net/?nick=wikiuser&channels=faraday-dev&prompt=1&uio=d4)
* Screenshots: https://github.com/infobyte/faraday/wiki/Screenshots
* Send your ideas and suggestions here: [https://www.faradaysec.com/ideas](https://www.faradaysec.com/ideas)
## Presentations
* Ekoparty Security Conference - 2017:
* http://blog.infobytesec.com/2017/10/ekoparty-2017-review_23.html
* Black Hat Arsenal Asia - 2017:
* https://www.blackhat.com/asia-17/arsenal.html#faraday
* Zero Nights - 2016
* https://www.slideshare.net/AlexanderLeonov2/enterprise-vulnerability-management-zeronights16
* AV Tokio - 2016:
* http://en.avtokyo.org/avtokyo2016/event
* Black Hat Arsenal USA - 2016:
* https://www.blackhat.com/us-16/arsenal.html#faraday
* Black Hat Arsenal Europe - 2016
* https://www.blackhat.com/eu-16/arsenal.html#faraday
* SecurityWeekly - 2016:
* http://securityweekly.com/2016/08/02/security-weekly-475-federico-kirschbaum/
* Bsides Latam - 2016:
* http://www.infobytesec.com/down/Faraday_BsideLatam_2016.pdf
* Black Hat Arsenal Asia - 2016:
* https://www.blackhat.com/asia-16/arsenal.html#faraday
* Black Hat Arsenal Europe - 2015:
* https://www.blackhat.com/eu-15/arsenal.html#faraday
* Black Hat Arsenal USA - 2015:
* https://www.blackhat.com/us-15/arsenal.html#faraday
* http://blog.infobytesec.com/2015/08/blackhat-2015_24.html
* RSA - 2015:
* http://www.rsaconference.com/events/us15/expo-sponsors/exhibitor-list/1782/infobyte-llc
* http://blog.infobytesec.com/2015/05/infobyte-en-la-rsa-2015.html
* Ekoparty Security Conference - 2014:
* https://www.youtube.com/watch?v=_j0T2S6Ppfo
* Black Hat Arsenal - 2011
* http://www.infobytesec.com/down/Faraday_BH2011_Arsenal.pdf
* Ekoparty Security Conference - 2010:
* http://prezi.com/fw46zt6_zgi8/faraday/
* http://vimeo.com/16516987
IMPORTANT
===========
Please be kind to remove all your pyc files before running faraday if you are updating this piece of software.
Make sure you run ```./faraday.py --update``` the first time after an update!
New features in the latest update
=====================================
September 21, 2018:
* Fix bug: manage.py status_check
* Fix bug: manage.py initdb
September 17, 2018:
---
* Fix get exploits API
* New searcher feature
* Added host_os column to status report
* Fix and error while trying to execute server with --start
* Added option --choose-password to initdb
* Continous scan updated for Nessus 7
* Refactor on server.config to remove globals
* Added a directory for custom templates for executive reports (pro and corp)
* Activity feed shows more results and allows to filter empty results
* Allow ot create workspace that start with numbers
* Added more variables to executive reports (pro and corp)
* Fixed some value checking on tasks api (date field)
* OpenVas plugin updated
* Appscan plugin update
* Added no confirmed vulns to report api
* Fixed a bug on workspace API when the workspace already exists on database
* Fix owner filter on status report
* Fixes on import_csv fplugin when the api returned 409
* Fixes on status_check
* Fixed a bug on webui when workspace permission was changed (pro and corp)
* Update nexpose plugin
* uigrid library updated to latest version
* Bug fix on plugin automatic detection
* Fixed a bug on executive reports when multiple reports were scheduled
* Avoid closing the executive report and new vuln modal when the form has data
* Status report open new tab for evidence
* added change_password to manage.py
* Update wapiti plugin
* Fixed vuln count on executive report (pro and corp)
* Fixed css align in some tables
* Fixed No ports available error on the client
August 17, 2018:
---
* Updated code to use Flask 1.0
* Add threadfix integration (corp only)
* Fix create_service fplugin
* Executive report bug fix on tags
* Persistence server bug fix on impact and ease of resolution
* Fix unicode error bug on executive reports
* Updated code to support latest Twisted version
* Updated all requirements to use >=
* Fix dry run on create_host fplugin
* Fixed del_all_vulns_with and del_all_hosts
* Improved executive reports status update refresh
* Websocket port is configurable now
* Change minimum font size in tag cloud
* Fixed a problem with shodan icon on dashboard
* Updated license check on deleted users
* Users with role client was not able to change password, bug fixed
* Updated code to support pip 10
* Added ldap to status check
* Credentials icon aligned
* Deamon now allows to execute faraday sever in more than one port and more than one process for multiplexation
* All views now check for permissions on workspace
* Pull requests #229, #231, #239 and #240 are merged
* Avoid polling deleted executive reports
* Added documentation to project
* Fix self xss on webshell
* Add postgres locks check on status_check
* Vuln counter fix when confirmed is on
July 26, 2018:
---
* Interface removed from model and from persistence server lib (fplugin)
* Performance iprovements on the backend
* Add quick change workspace name (from all views)
* Changed the scope field of a workspace from a free text input to a list of targets
* New faraday styles in all webui views
* Add search by id for vulnerabilities
* Add new plugin sslyze
* Add new plugin wfuzz
* Add xsssniper plugin
* Fix W3af, Zap plugins
* Add brutexss plugin
* Allow to upload report file from external tools from the web
* Fix sshcheck import file from GTK
* Add reconng plugin
* Add sublist3r plugin
* Add HP Webinspect plugin
* Add dirsearch plugin
* Add ip360 plugin
* CouchDB was replaced by PostgreSQL :)
* Host object changed, now the name property is called ip
* Interface object was removed
* Note object was removed and replaced with Comment
* Communication object was removed and replaced with Comment
* Show credentials count in summarized report on the dashboard
* Remove vuln template CWE fields, join it with references
* Allow to search hosts by hostname, os and service name
* Allow the user to specify the desired fields of the host list table
* Add optional hostnames, services, MAC and description fields to the host list
* Workspace names can be changed from the Web UI
* Exploitation and severity fields only allow certain values. CWE CVEs were fixed to be valid. A script to convert custom CSVs was added.
* Web UI path changed from /_ui/ to / (_ui has now a redirection to / for keeping backwards compatibility)
* dirb plugin creates an informational vulnerability instead of a note.
* Add confirmed column to exported csv from webui
* Fixes in Arachni plugin
* Add new parameters --keep-old and --keep-new for faraday CLI
* Add new screenshot fplugin which takes a screenshot of the ip:ports of a given protocol
* Add fix for net sparker regular and cloud fix on severity
* Removed Chat feature (data is kept inside notes)