Ensure that if the delimiter is set other than comma then download the file as '.txt' file. Fixes #4573

This commit is contained in:
Nagesh Dhope 2020-04-17 17:37:02 +05:30 committed by Akshay Joshi
parent 52d031f2d0
commit 8bdfa18efb
32 changed files with 131 additions and 46 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 172 KiB

After

Width:  |  Height:  |  Size: 179 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 114 KiB

After

Width:  |  Height:  |  Size: 185 KiB

BIN
docs/en_US/images/preferences_browser_nodes.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 127 KiB

BIN
docs/en_US/images/preferences_browser_properties.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 100 KiB

BIN
docs/en_US/images/preferences_dashboard_display.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 135 KiB

BIN
docs/en_US/images/preferences_dashboard_graphs.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 97 KiB

After

Width:  |  Height:  |  Size: 135 KiB

BIN
docs/en_US/images/preferences_debugger_display.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 101 KiB

After

Width:  |  Height:  |  Size: 110 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 77 KiB

BIN
docs/en_US/images/preferences_paths_binary.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 142 KiB

BIN
docs/en_US/images/preferences_paths_help.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 130 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 79 KiB

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 79 KiB

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 100 KiB

After

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 141 KiB

After

Width:  |  Height:  |  Size: 163 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 184 KiB

After

Width:  |  Height:  |  Size: 217 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 104 KiB

After

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 143 KiB

After

Width:  |  Height:  |  Size: 166 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

After

Width:  |  Height:  |  Size: 89 KiB

BIN
docs/en_US/images/preferences_storage_options.png Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 62 KiB

After

Width:  |  Height:  |  Size: 98 KiB

View File

@ -247,14 +247,14 @@ Use the fields on the *Auto Completion* panel to set the auto completion options
:alt: Preferences dialog sqleditor csv output option
:align: center
Use the fields on the *CSV Output* panel to control the CSV output.
Use the fields on the *CSV/TXT Output* panel to control the CSV/TXT output.
* Use the *CSV field separator* drop-down listbox to specify the separator
character that will be used in CSV output.
character that will be used in CSV/TXT output.
* Use the *CSV quote character* drop-down listbox to specify the quote character
that will be used in CSV output.
that will be used in CSV/TXT output.
* Use the *CSV quoting* drop-down listbox to select the fields that will be
quoted in the CSV output; select *Strings*, *All*, or *None*.
quoted in the CSV/TXT output; select *Strings*, *All*, or *None*.
* Use the *Replace null values with* option to replace null values with
specified string in the output file. Default is set to 'NULL'.

View File

@ -122,7 +122,7 @@ You can:
* Select and copy from the displayed result set.
* Use the *Execute/Refresh* options to retrieve query execution information and
set query execution options.
* Use the *Download as CSV* icon to download the content of the *Data Output*
* Use the *Download as CSV/TXT* icon to download the content of the *Data Output*
tab as a comma-delimited file.
* Edit the data in the result set of a SELECT query if it is updatable.

View File

@ -177,7 +177,8 @@ Query Execution
| | | |
| | * Select *Clear History* to erase the content of the *History* tab. | |
+----------------------+---------------------------------------------------------------------------------------------------+----------------+
| *Download as CSV* | Click the *Download as CSV* icon to download the result set of the current query to a | F8 |
| | comma-separated list. You can specify the CSV settings through | |
| | *Preferences -> SQL Editor -> CSV output* dialogue. | |
| *Download as CSV/TXT*| Click the *Download as CSV/TXT* icon to download the result set of the current query as a *.csv* | F8 |
| | or as a *.txt* file. if *CSV field seperator* set to comma(,) else as a *.txt* file. | |
| | You can specify the CSV/TXT settings through *Preferences -> SQL Editor -> CSV/TXT output* | |
| | dialogue. | |
+----------------------+---------------------------------------------------------------------------------------------------+----------------+

View File

@ -37,6 +37,7 @@ Bug fixes
| `Issue #4440 <https://redmine.postgresql.org/issues/4440>`_ - Ensure the DROP statements in reverse engineered SQL are properly quoted for all objects.
| `Issue #4445 <https://redmine.postgresql.org/issues/4445>`_ - Ensure all object names in the title line of the reverse-engineered SQL are not quoted.
| `Issue #4512 <https://redmine.postgresql.org/issues/4512>`_ - Fixed calendar opening issue on the exception tab inside the schedules tab of pgAgent.
| `Issue #4573 <https://redmine.postgresql.org/issues/4573>`_ - Ensure that if the delimiter is set other than comma then download the file as '.txt' file.
| `Issue #4684 <https://redmine.postgresql.org/issues/4684>`_ - Fixed encoding issue while saving data in encoded charset other than 'utf-8'.
| `Issue #4709 <https://redmine.postgresql.org/issues/4709>`_ - Added schema-qualified dictionary names in FTS configuration to avoid confusion of duplicate names.
| `Issue #4856 <https://redmine.postgresql.org/issues/4856>`_ - Enable the save button by default when a query tool is opened with CREATE or other scripts.

View File

@ -83,11 +83,11 @@ let queryToolActions = {
}
if (!sqlQuery) return;
let filename = 'data-' + new Date().getTime() + '.csv';
let extension = sqlEditorController.preferences.csv_field_separator === ',' ? '.csv': '.txt';
let filename = 'data-' + new Date().getTime() + extension;
if (!sqlEditorController.is_query_tool) {
filename = sqlEditorController.table_name + '.csv';
filename = sqlEditorController.table_name + extension;
}
sqlEditorController.trigger_csv_download(sqlQuery, filename);

View File

@ -114,9 +114,9 @@ function updateUIPreferences(sqlEditor) {
$el.find('#btn-download')
.attr('title',
shortcut_title(gettext('Download as CSV'),preferences.download_csv))
shortcut_title(gettext('Download as CSV/TXT'),preferences.download_csv))
.attr('aria-label',
shortcut_title(gettext('Download as CSV'),preferences.download_csv));
shortcut_title(gettext('Download as CSV/TXT'),preferences.download_csv));
$el.find('#btn-save-data')
.attr('title',

View File

@ -1329,14 +1329,18 @@ def start_query_download_tool(trans_id):
field_separator=blueprint.csv_field_separator.get(),
replace_nulls_with=blueprint.replace_nulls_with.get()
),
mimetype='text/csv'
mimetype='text/csv' if
blueprint.csv_field_separator.get() == ','
else 'text/plain'
)
if 'filename' in data and data['filename'] != "":
filename = data['filename']
else:
import time
filename = str(int(time.time())) + ".csv"
filename = '{0}.{1}'. \
format(int(time.time()), 'csv' if blueprint.
csv_field_separator.get() == ',' else 'txt')
# We will try to encode report file name with latin-1
# If it fails then we will fallback to default ascii file name

View File

@ -7,6 +7,7 @@
# This software is released under the PostgreSQL Licence
#
##########################################################################
from unittest.mock import patch
from pgadmin.utils.route import BaseTestGenerator
from pgadmin.browser.server_groups.servers.databases.tests import utils as \
@ -32,7 +33,9 @@ class TestDownloadCSV(BaseTestGenerator):
output_columns='"A","B","C"',
output_values='1,2,3',
is_valid_tx=True,
is_valid=True
is_valid=True,
download_as_txt=False,
filename='test.csv'
)
),
(
@ -44,7 +47,9 @@ class TestDownloadCSV(BaseTestGenerator):
output_columns=None,
output_values=None,
is_valid_tx=False,
is_valid=False
is_valid=False,
download_as_txt=False,
filename='test.csv'
)
),
(
@ -56,7 +61,37 @@ class TestDownloadCSV(BaseTestGenerator):
output_columns=None,
output_values=None,
is_valid_tx=True,
is_valid=False
is_valid=False,
download_as_txt=False,
filename='test.csv'
)
),
(
'Download as txt without filename parameter',
dict(
sql='SELECT 1 as "A",2 as "B",3 as "C"',
init_url='/datagrid/initialize/query_tool/{0}/{1}/{2}/{3}',
donwload_url="/sqleditor/query_tool/download/{0}",
output_columns='"A";"B";"C"',
output_values='1;2;3',
is_valid_tx=True,
is_valid=True,
download_as_txt=True,
filename=None
)
),
(
'Download as csv without filename parameter',
dict(
sql='SELECT 1 as "A",2 as "B",3 as "C"',
init_url='/datagrid/initialize/query_tool/{0}/{1}/{2}/{3}',
donwload_url="/sqleditor/query_tool/download/{0}",
output_columns='"A","B","C"',
output_values='1,2,3',
is_valid_tx=True,
is_valid=True,
download_as_txt=False,
filename=None
)
),
]
@ -95,30 +130,62 @@ class TestDownloadCSV(BaseTestGenerator):
url = self.donwload_url.format(self.trans_id)
# Disable the console logging from Flask logger
self.app.logger.disabled = True
response = self.tester.post(
url,
data={"query": self.sql, "filename": 'test.csv'}
)
# Enable the console logging from Flask logger
self.app.logger.disabled = False
if self.is_valid:
# when valid query
self.assertEquals(response.status_code, 200)
csv_data = response.data.decode()
self.assertTrue(self.output_columns in csv_data)
self.assertTrue(self.output_values in csv_data)
elif not self.is_valid and self.is_valid_tx:
# When user enters wrong query
self.assertEquals(response.status_code, 200)
response_data = json.loads(response.data.decode('utf-8'))
self.assertFalse(response_data['data']['status'])
self.assertTrue(
'relation "this_table_does_not_exist" does not exist' in
response_data['data']['result']
)
if self.filename is None:
if self.download_as_txt:
with patch('pgadmin.tools.sqleditor.blueprint.'
'csv_field_separator.get', return_value=';'), patch(
'time.time', return_value=1587031962.3808076):
response = self.tester.post(url, data={"query": self.sql})
headers = dict(response.headers)
# when valid query
self.assertEquals(response.status_code, 200)
csv_data = response.data.decode()
self.assertTrue(self.output_columns in csv_data)
self.assertTrue(self.output_values in csv_data)
self.assertIn('text/plain', headers['Content-Type'])
self.assertIn('1587031962.txt',
headers['Content-Disposition'])
else:
with patch('time.time', return_value=1587031962.3808076):
response = self.tester.post(url, data={"query": self.sql})
headers = dict(response.headers)
# when valid query
self.assertEquals(response.status_code, 200)
csv_data = response.data.decode()
self.assertTrue(self.output_columns in csv_data)
self.assertTrue(self.output_values in csv_data)
self.assertIn('text/csv', headers['Content-Type'])
self.assertIn('1587031962.csv',
headers['Content-Disposition'])
else:
# when TX id is invalid
self.assertEquals(response.status_code, 500)
response = self.tester.post(
url,
data={"query": self.sql, "filename": self.filename}
)
headers = dict(response.headers)
# Enable the console logging from Flask logger
self.app.logger.disabled = False
if self.is_valid:
# when valid query
self.assertEquals(response.status_code, 200)
csv_data = response.data.decode()
self.assertTrue(self.output_columns in csv_data)
self.assertTrue(self.output_values in csv_data)
self.assertIn('text/csv', headers['Content-Type'])
self.assertIn(self.filename, headers['Content-Disposition'])
elif not self.is_valid and self.is_valid_tx:
# When user enters wrong query
self.assertEquals(response.status_code, 200)
response_data = json.loads(response.data.decode('utf-8'))
self.assertFalse(response_data['data']['status'])
self.assertTrue(
'relation "this_table_does_not_exist" does not exist' in
response_data['data']['result']
)
else:
# when TX id is invalid
self.assertEquals(response.status_code, 500)
database_utils.disconnect_database(self, self._sid, self._did)

View File

@ -211,7 +211,7 @@ def RegisterQueryToolPreferences(self):
self.csv_quoting = self.preference.register(
'CSV_output', 'csv_quoting',
gettext("CSV quoting"), 'options', 'strings',
category_label=gettext('CSV Output'),
category_label=gettext('CSV/TXT Output'),
options=[{'label': gettext('None'), 'value': 'none'},
{'label': gettext('All'), 'value': 'all'},
{'label': gettext('Strings'), 'value': 'strings'}],
@ -224,7 +224,7 @@ def RegisterQueryToolPreferences(self):
self.csv_quote_char = self.preference.register(
'CSV_output', 'csv_quote_char',
gettext("CSV quote character"), 'options', '"',
category_label=gettext('CSV Output'),
category_label=gettext('CSV/TXT Output'),
options=[{'label': '"', 'value': '"'},
{'label': '\'', 'value': '\''}],
select2={
@ -236,7 +236,7 @@ def RegisterQueryToolPreferences(self):
self.csv_field_separator = self.preference.register(
'CSV_output', 'csv_field_separator',
gettext("CSV field separator"), 'options', ',',
category_label=gettext('CSV output'),
category_label=gettext('CSV/TXT output'),
options=[{'label': ';', 'value': ';'},
{'label': ',', 'value': ','},
{'label': '|', 'value': '|'},
@ -250,7 +250,7 @@ def RegisterQueryToolPreferences(self):
self.replace_nulls_with = self.preference.register(
'CSV_output', 'csv_replace_nulls_with',
gettext("Replace null values with"), 'text', 'NULL',
category_label=gettext('CSV output'),
category_label=gettext('CSV/TXT output'),
help_str=gettext('Specifies the string that represents a null value '
'while downloading query results as CSV. You can '
'specify any arbitrary string to represent a '

View File

@ -298,13 +298,22 @@ describe('queryToolActions', () => {
}));
});
it('calls trigger_csv_download with the query and the filename', () => {
it('calls trigger_csv_download with the query and the filename with .csv extension', () => {
let filename = 'data-' + time + '.csv';
queryToolActions.download(sqlEditorController);
expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(selectedQueryString, filename);
});
it('calls trigger_csv_download with the query and the filename with .txt extension', () => {
sqlEditorController.preferences.csv_field_separator = ';';
let filename = 'data-' + time + '.txt';
queryToolActions.download(sqlEditorController);
expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(selectedQueryString, filename);
});
});
describe('when there is no selection', () => {
@ -622,6 +631,9 @@ describe('queryToolActions', () => {
table_name: 'iAmATable',
is_query_tool: true,
check_data_changes_to_execute_query: jasmine.createSpy('check_data_changes_to_execute_query'),
preferences: {
csv_field_separator: ',',
},
};
}
});