- fixed gettext usage with .format() only for original text with %s
- fixed typos
- fixed translation yes/no buttons in the dialog.
- improved translating sentences without "connecting" words (eg. see web/pgadmin/dashboard/static/js/dashboard.js,
word 'cancel' needs to be translated in the Czech language as 'zrušit' but in another sentence as 'zrušení')
- added gettext for text translations.
1) Upgraded passlib==1.7.1 to passlib==1.7.2
2) Replace unmaintained Flask-Security with maintained Flask-Security-Too package, which is also compatible with python 3.8
3) Other compatibility code changes.
Currently supported objects are Table, View, Materialized View, Function and Procedure.
Backend comparison of two schemas implemented by: Akshay Joshi
Fixes#3452.
2) Fix server connection drops out issue in query tool. Fixes#4818
3) Fix VPN network disconnect issue where pgAdmin4 hangs on expanding the Servers node. Fixes#4926.
4) Ensure that the Servers collection node should expand independently of server connections. Fixes#4933.
Set the default connection timeout to 10 seconds instead of 0.
1) Remove duplicate logic.
2) Child node specific logic should be inside the child node and table node gets the required information from the respective child node.
Note:- Refactoring of columns node still in progress.
Fixes#4807
tree toggle issue
Query tool inteliSence issue eg. when there is only one option and drop down is not shown
Backup and restore windows locator changes
Fixes required due to resolving rm # 4041
Dependent tab not showing data sometime, so refreshed the page and handled it
Due to change of logic for auto commit, did the required changes
Due to fix of RM 4062, did the required workaround which broke the test case.
instead of 'UPDATE' statement, which is not allowed in the the read-only
instances.
Reported by: Robert J. Rotter
Solution provided by: Tom Lane
Fixes#3973
As sessions in pgAdmin4 are filesystem based session, they need locking
for avoiding the access from multiple threads, specially running as an
WSGI application.
Fixes#3547
- Extract SQLEditor.execute and SQLEditor._poll into their own files and add test around them
- Extract SQLEditor backend functions that start executing query to their own files and add tests around it
- Move the Explain SQL from the front-end and now pass the Explain plan parameters as a JSON object in the start query call.
- Extract the compile_template_name into a function that can be used by the different places that try to select the version of the template and the server type
Because - the get_locale(...) function was being executed before any
request, and user login after first request. Hence - the values for
the labels in the Preferences are not being translated properly.
Fetch the raw value of user_language from configuration in case of
runtime/non-server mode to fix the issue.
This issue was caused because we recently added session_write_delay in session.
So session won't be written/updated to disk from memory until specified seconds are elapsed.
However we must forcefully write/update session to disk if user loges in or out irrespective of session_write_delay to keep sessions from memory and disk in sync as user logged in status is kept in session.
Session object is updated, everytime a request is being served, and
that was forcing the session object dumped on the dist on every request.
On windows, it was causing issues on slower system on startup. Because -
windows file system locks the file, when it is opened by any
application. And, frequent requests on the pgAdmin main UI rendering
was causing issues, because of that.
In order to resolve the issue, we will not write the session data on
disk for every request, but - only after certain delay (in seconds),
from it was last written. It can be configured using the attribute
'PGADMIN_SESSION_DISK_WRITE_DELAY' in the configuration file,
default vaule for the delay is 10.
(i.e. 10 seconds)
This allows the user to create partitioned table and it's partitions.
- Edit partitions, Create/Listing of constraints, rules, triggers under partitions. (Implemented by Harshal)
- Updating browser tree node when create/attach/detach partitions from table dialog.(Implemented by Ashesh)
Created a url_for function module to be used by different javascript
modules to use the same functionality of flask at client side.
A python module can expose the list of endpoints, used by its
javascripts, by overriding the 'get_exposed_url_endpoints(self)'
function.
In this patch, we have modified all the browser modules to use this
function. It will allow us to move the majority of the javascript
modules of browser nodes from templates directory to the static
directory.
TODO::
- Move these javascripts modules to the static directory.
- Use this function in all the applicable javascript modules.
e.g. tools, miscellaneous modules
1) To handle non-ascii filenames which we set from table name. Fixes#2314
2) To handle non-ascii query data. Fixes#2253
3) To dump JSON type columns properly in csv. Fixes#2360
In order to resolve the non-ascii characters in path (in user directory,
storage path, etc) on windows, we have converted the path into the
short-path, so that - we don't need to deal with the encoding issues
(specially with Python 2).
We've resolved majority of the issues with this patch.
We still need couple issues to resolve after this in the same area.
TODO
* Add better support for non-ascii characters in the database name on
windows with Python 3
* Improve the messages created after the background processes by
different modules (such as Backup, Restore, Import/Export, etc.),
which does not show short-paths, and xml representable characters for
non-ascii characters, when found in the database objects, and the file
PATH.
Fixes#2174, #1797, #2166, #1940
Initial patch by: Surinder Kumar
Reviewed by: Murtuza Zabuawala
Using 'psycopg2.extensions.UNICODE' (for Python < 3) in the psycopg2
driver for proper conversation of unicode characters. Also - adjusted
the string typecaster to take care of different character types (char,
character, text, name, character varying, and their array types).
Reviewed by: Dave Page, Murtuza Zabuawala & Akshay Joshi
- don't spin up app and chromedriver between each test
- catching signals also tears down the app
- do layout reset between tests, but assume that tests will not leave a modal opened.
Use selenium built-in waiting function and fix flakiness around clicking the alertify OK button
- we think the OK button does not have its event bound when it is created.
If you see more flakiness around clicking the alertify OK button, let us know. The element is clickable but we have to arbitrarily wait for the event to be bound and that timing may vary system to system.
The feature tests are about 7 seconds faster now.
Tira & Joao
To chase down a set of tests that were failing together, we wanted to run
deeper trees of tests. We changed the behavior of the --pkg argument to
runtests.py so it runs tests under a package and its subpackages, instead
of for a single node.
It gave us more flexibility in what tests to run. This also includes the
--exclude option.
To run tests for a single node, ".tests" can be appended to the package path.
Tira & George.
1. Code changes (due to drop objects functionality).
2. Quoting for database names in drop databases.
3. Code changes for import errors for pickle_path and advanced_config variables.
Made backend changes for:
* Taking care of the connection status in the psycopg2 driver. And, when
the connection is lost, it throws a exception with 503 http status
message, and connection lost information in it.
* Allowing the flask application to propagate the exceptions even in the
release mode.
* Utilising the existing password (while reconnection, if not
disconnected explicitly).
* Introduced a new ajax response message 'service_unavailable' (http
status code: 503), which suggests temporary service unavailable.
Client (front-end) changes:
* To handle the connection lost of a database server for different
operations by generating proper events, and handle them properly.
Removed the connection status check code from different nodes, so that
- it generates the proper exception, when accessing the non-alive
connection.
Fixes#1387
1. The user will specify the tablespace path in test_config.json.in
2. If tablespace path not found, skip the test cases for that server(Only tablespace test cases)
3. Add the skipped test summary in the test result. (Now it's showing on console + in log file, but need to update in a final enhanced test summary report. Which is research point we will work on that after finishing all nodes API test cases)
4. Removed the test_ prefix from the values in the config files.
5. Add tablespace and roles tests
- Test framework support API testing with multiple server for this we need to modify test_config.json(for user it’s test_config.json.in) and test_advanced_config.json(for user it’s test_advanced_config.json.in). Server details of PG and PPAS are included in both .in files.
- Removed the logic of logging in the test client on each test scenario(As per Khushboo's comment in previous email). We need this logic in test cases under ‘browser/tests/’ as for test scenarios like change password and invalid login test cases as test client should be logged out first. So, as per this the code is slightly modified in ‘browser/tests/’.
optimized to work from multiple threads. It has too many frequent
transaction from multiple threads, and that tends to result into the
'database is locked' error of sqlite.
With the new implemenation, we're using the caching mechanism, which
keep the data in the memory all the time, and saves it on request
completion, and loads it only for the first time. Also, it will storage
the data using pickle, which will be faster than accessing sqlite.
Fixes#1329
1) No handling for INTERVAL type datetime.
For example: executing query
SELECT INTERVAL '15 minutes';
throws json serialization error, because it returns time in timedelta format which is not handled.
Added support to handle timedelta datetime format in DataTypeJSONEncoder class
2) When we try to get BC dates from database raises ValueError: year is out of range
For eg:
SELECT TIMESTAMP '0044-03-15 10:00:00 BC',
It is because pyscopg2 doesn't handle BC datetime format.
So we have defined our method which type cast the datetime value to string in pyscopg2 overriding default behaviour.
Reference:
http://initd.org/psycopg/docs/advanced.html#type-casting-from-sql-to-pythonccf3693be6/pgcli/pgexecute.py
1) Unable to select sql file through query tool on windows OS. In file_manager.js, we are stripping initial slash '/' from the path obtained, but we should not strip if it is full path like 'c:/path/to/dir/filename.ext'
2) Handle directory path if STORAGE_DIR is None. Proper checks are added.
As we convert the binary password to string during storing the
connection information, we also need to convert it back to byte-arrays
during restoring the connections.
Hence - decode the passowrd using 'utf-8' encoding during storing the
connection information, and encode it during restoring the connections.
Tweaked by Ashesh Vashi to integrate the backgroud process, and also
with some improvements as stated below:
* Resolved an issue loading existing preference.
* Improved the background process observer/executor for supporting
detalied view.
* Added the utility path preferences in the ServerType class.
operation like backup, restore, etc within it.
Also:
* improvised the color combination of the background process logger.
* Removed an unnecessary print statement from the
get_storage_directory(..) function, also return None if STORAGE_DIR
is set to None.
selection, creation, upload/download files/directories resides on the
server side.
This will be useful for file selection/creation for different server
side utilites like pg_dump, pg_dumpall, pg_restore.
server.
Current implementation keeps the cursor object in the 'g' (current
request context) object identified by the connection-id.
But - it fails to identify a different connection object request, when
we have same database name in different database server, it does not
able to identify it as separate request, Hence - now we will use
server-id qualified identifier for the same object.
Thanks Neel Patel for pointing out the issue.
required by its javascript module.
This will allow us to load the javascript modules as a static file, and
not as a Jinja2 template. This will increase the load time, as it will
decrease number of templates to be processed during loading those
javascripts.
DriverRegistry is executed second time.
Also, initialize the driver before registering different blueprints,
which uses those driver inside them.
Thanks Khushboo for reporting the issue.
At the moment, we will only allow to fetch status messages from the
asynchronous connection only, later - we may implement to fetch the
status message from the normal connection too.
configuration database due to unable to identify the change properly.
Thanks Dave for reporting.
This also includes a support to use a 'text' type of preference, and
show help string next to the control.
This commit takes care of the following issues/improvements.
* Adding missing imports for unauthorised in database module
* Node under Servers Nodes (i.e. Databases, tablespaces, roles nodes)
need to be inherited from PGChildNodeView (and, not from NodeView) for
adding server version check for their children.
* Adding statistics for database, and tablespaces in its node (not, yet
in UI)
* Renaming the camel case methods with proper name.
(i.e. getSQL -> get_sql, getNewSQL -> get_new_sql, etc.)
* Fixed the functions going beyond the text limit (column: 80) in
Databases, Roles & Tablespaces modules.
* Fixed the node method of Database module, which was not tested ever.
* We do not need separate SQL template for fetching the name (i.e.
get_name.sql), using the 'nodes.sql' for the same.
* Optimise the query for fetching ACLs for the database node, we didn't
require to join certain tables, while fetching only the ACLs.
* Introduced the list of the ACLs (regular and default ACLs for
different type) supported by each version [Databases Module].
* Renamed the templates 'get_nodes.sql' to' nodes.sql' to make it
consistent with other modules.
* Removed the checks for the authentication table use, as we don't need
to expose the password to the users (even the encrypted MD5). Using
the pg_roles view always for fetching roles/users information now.
* Resolved some typos in unreachable (specially the exceptions
catchment area.)
* Logging the exception in the application.
* Using qtLiteral, qtIdent properly in the templates (do not assume
about the types of data.)
* Using tsid as identifier instead of did for the tablespaces.
* Using nodes method of tablespace view for fetching individual node
information.
* Removing the hardcoded node information from the 'parse_priv_to_db'
function, and pass on allowed ACLs by the caller nodes.
* Using 'nodes.sql' to fetch name of the template instead of writing
that in the delete.sql template, which is definitely wrong place to
fetch the name of the object. [Tablespace Module]
of double quote within it.
The 'replace' function of immutable type string do create a new
instance of type string, which needed to be reassgined to the variable
'value'.
Reported By: Murtuza Zabuawala
It was due to wrong index used for the COL_NAME_KEYWORD, used by the
'needsQuoting' function.
Also, replaced a tab with a space in the generate_keyword.py file.
Reported by: Sanket Mehta, Khushboo Vashi
with schema specified object name. This will also works with
EntepriseDB's package too.
Also, added dependency on the current connection object, it will allow
us to use the keywords list specific to the database server in futuer.
You can use the qtIdent template filter as:
conn|qtIdent(name)
or,
conn|qtIdent(schema_name, name)
