pgadmin4/web/regression/README
2020-01-03 19:00:08 +05:30

253 lines
9.0 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

pgAdmin 4 Test Framework
========================
This regression framework is designed to execute unit tests for all modules to
help catch regressions in the code.
Assumptions
-----------
- pgAdmin must have SMTP configured if SERVER_MODE == True.
- There should be a single server group present in the browser.
- Currently each module will have testcases related to ONLY GET, POST, PUT,
and DELETE apis.
Environment
-----------
Regression tests should be run in a Python environment that has all the
pre-requisite packages installed from $PGADMIN4_SRC/requirements.txt. There
are some additional dependencies for running the test suite; these can be
installed with:
(pgadmin4) $ pip install -r $PGADMIN4_SRC/web/regression/requirements.txt
While running in Linux environments install:
(pgadmin4) $ sudo apt-get install xsel
Otherwise the following error happens:
"Pyperclip could not find a copy/paste mechanism for your system"
General Information
-------------------
1) The required test cases should be placed under the /tests directory of the
respective module.
- 'pgadmin/browser/tests' directory contains test-cases for following
modules:
1. Login Page
2. Reset Password Page
3. Change Password Page
4. Logout Page
- 'pgAdmin4/web/pgadmin/browser/server_groups/tests/' shows an example of
tree traversal of the pgAdmin modules and how the test folder is required
for each individual module.
- 'pgadmin/browser/server_groups/servers/tests/' directory will have separate
file for each test-case:
1. test_server_add.py
2. test_server_delete.py
3. test_server_get.py
4. test_server_update.py
2) The pgAdmin4 source tree includes 2 different configuration file templates.
One file template for the server configuration
named test_config.json.in' and another for test configuration named
'test_advanced_config.json.in' in the pgAdmin4/web/regression directory.
After completing the pgAdmin4 configuration, you must make a working copy of
the templates called test_config.json and test_advance_config.json
before modifying the file contents.
2a) The following command copies the test_config.json.in file, creating a
configuration file named test_config.json (same way user can copy
test_advance_config.json.in file into test_advance_config.json)
$ cp pgadmin4/web/regression/test_config.json.in \
pgadmin4/web/regression/test_config.json
2b) After creating the server and test configuration file, add (or modify)
parameter values as per requirements. The pgAdmin4 regression framework
expects to find the files in the directory
'/<installation dir>/web/regression/'. If you move the file to another
location, you must create a symbolic link that specifies the new location.
2c) Specifying Server Configuration file:
Server details and connection properties as per their local setup. The
test_config file is in JSON format and property values are
case-sensitive.
2d) Specifying the Test Configuration file:
The user can add/change test data as per their need. The
test_advanced_config file is in JSON format and property values are
case-sensitive.
Test Data Details
-----------------
"pgAdmin4 Login Credentials" (used to login to pgAdmin):
login_username = login id
login_password = login password
new_password = new login password
"pgAdmin4 Test User Credentials" (dummy user used for testing user management):
login_username = login id
login_password = login password
new_password = new login password
"server_credentials":
name = Server/database Name
db_username = Database Username
host = IP Address of Server
db_password = Database Password
db_port = Database Port
maintenance_db = Maintenance Database
sslmode = SSL Mode
comment = Any Comments to add
tablespace_path = A path under which a tablespace can be created
Execution:
-----------
Python Tests:
- For feature tests to run as part of the entire test suite, Chrome and
chromedriver need to be installed; get chromedriver from
https://sites.google.com/a/chromium.org/chromedriver/downloads or a
package manager and make sure it is in the PATH
- For feature tests to run on Firefox, geckodriver need to be installed;
- Get geckodriver from https://github.com/mozilla/geckodriver/releases.
- Extract the binary and run chmod +x geckodriver.
- Copy geckodriver into /usr/local/bin or make sure path of the
geckodriver must be specified in the PATH.
- Set the "default_browser" parameter in test_config file or pass the command
line option --default_browser. Supported browsers are "Chrome" and
"Firefox".
- The test framework is modular and pluggable and dynamically locates tests
for modules which are discovered at runtime. All test cases are found
and registered automatically by its module name in
'pgadmin4/web/pgadmin/utils/test.py' file.
- Change to the regression test directory:
run 'cd web/regression'
- Execute the test framework for all nodes
run 'python runtests.py --pkg all' or just:
'python runtests.py'
- Execute test framework for entire package
Example 1) Run test framework for 'browser' package
run 'python runtests.py --pkg browser'
Example 2) Run test framework for 'database' package
run 'python runtests.py --pkg browser.server_groups.servers.databases'
Example 3) Run feature tests
run 'python runtests.py --pkg feature_tests
- Execute test framework for single node at a time
Example 1) Run test framework for 'browser' node
run 'python runtests.py --pkg browser.server_groups.tests'
Example 2) Run test framework for 'database' node
run 'python runtests.py --pkg browser.server_groups.servers.databases.tests'
- Execute test framework for certain modules of a test pkg
Example 1) Run test framework for 'sqleditor' package and test_start_running_query module
run 'python runtests.py --pkg tools.sqleditor --modules test_start_running_query'
Example 2) Run test framework for 'sqleditor' package and test_start_running_query,test_query_tool_fs_utils modules
run 'python runtests.py --pkg tools.sqleditor --modules test_start_running_query,test_query_tool_fs_utils'
- Exclude a package and its subpackages when running tests:
Example: exclude feature tests but run all others:
run 'python runtests.py --exclude feature_tests'
Example: exclude multiple packages:
run 'python runtests.py --exclude browser.server_groups.servers.databases,browser.server_groups.servers.tablespaces'
- Execute reverse engineered SQL test framework
Example 1) Execute only reverse engineered sql test framework for all nodes
run 'python runtests.py --pkg resql'
Example 2) Execute only reverse engineered SQL test framework for some modules
run 'python runtests.py --pkg resql --modules sequences,functions'
Code Coverage:
---------------
- Test framework is able to calculate the code coverage.
- Coverage package(coverage) is added in $PGADMIN4_SRC/web/regression/requirements.txt file
How to generate code coverage report for API test-suite?
-------------------------------------
- Change to the regression test directory:
run 'cd $PGADMIN4_SRC/web/regression'
- Before running code coverage we need to configure 'regression/.coveragerc' file.
i). Create 'regression/.coveragerc' file.
ii). Copy content of 'regression/.coveragerc.in' to 'regression/.coveragerc'
iii). Modify 'regression/.coveragerc' file as per our need as 'regression/.coveragerc.in' has default configurations
In 'regression/.coveragerc' file we need to mention some parameters
like 'source'(project path),'include'(files/modules to be included for
coverage),'omit'(files/modules to be omitted from coverage)
We can also add more parameters according to our need.
For more info please read coverage.py's official document here
'http://coverage.readthedocs.io/en/coverage-4.2/install.html'
- Run coverage
With all modules
run 'python runtests.py --coverage --exclude feature_tests'
With specific module
run 'python runtests.py --pkg browser.server_groups.servers.tests --coverage'
- After execution of coverage, we will see code coverage report on console.
For a nicer presentation, '/regression/covhtml' directory gets created.
Open 'index.html' file in browser and you will see detail coverage report.
Javascript Tests:
- Install node.js and yarn, in the appropriate way for your platform. On macOS with Homebrew you might use:
brew install nodejs
brew install yarn
Whilst with MacPorts:
sudo port install nodejs7 yarn
- See also the top-level pgadmin/README : Bundling Javascript
- Javascript tests must be run from the web directory (since that is where node_modules and karma.conf reside):
cd web/
- Install the JS modules required for testing:
yarn install
- Now run the tests:
yarn run test:karma
yarn run test:karma-once